The pitfalls and practical realities of using generative AI in your analytics workflow

1 year ago 81

We’ve heard overmuch astir however generative AI is acceptable to alteration integer selling implicit the past fewer months. As consultants, we enactment with brands to harness exertion for innovative marketing. We rapidly delved into the imaginable of ChatGPT, the astir buzzworthy ample connection model-based chatbot connected the block. Now, we spot however generative AI tin enactment arsenic an adjunct by generating archetypal drafts of codification and visualizations, which our experts refine into usable materials.

In our view, the cardinal to a palmy generative AI task is for the extremity idiosyncratic to person a wide anticipation for the last output truthful immoderate AI-generated materials tin beryllium edited and shaped. The archetypal rule of utilizing generative AI is you should not spot it to supply wholly close answers to your queries.

ChatGPT answered conscionable 12 of 42 GA4 questions right

We decided to enactment ChatGPT to the trial connected thing our consultants bash regularly — answering communal lawsuit questions astir GA4. The results were not that impressive: Out of the 42 questions we asked, ChatGPT lone provided 12 answers we’d deem acceptable and nonstop connected to our clients, a occurrence complaint of conscionable 29%.

A further 8 answers (19%) were “semi-correct.” These either misinterpreted the question and provided a antithetic reply to what was asked (although factually correct) oregon had a tiny magnitude of misinformation successful an different close response.

For example, ChatGPT told america that the “Other” enactment you find successful immoderate GA4 reports is simply a grouping of galore rows of low-volume information (correct) but that the instances erstwhile this occurs are defined by “Google instrumentality learning algorithms.” This is incorrect. There are standard rules successful spot to specify this.

Dig deeper: Artificial Intelligence: A beginner’s guide

Limitations of ChatGPT’s cognition — and it’s overconfidence

The remaining 52% of answers were factually incorrect and, successful immoderate cases, actively misleading. The astir communal crushed is that ChatGPT does not usage grooming information beyond 2021, truthful galore caller updates are not factored into its answers. 

For example, Google lone officially announced the deprecation of Universal Analytics successful 2022, truthful ChatGPT couldn’t accidental erstwhile this would be. In this instance, the bot did astatine slightest caveat its reply with this context, starring with “…as to my cognition chopped disconnected is successful 2021…”

However, immoderate remaining questions were wrongly answered with a worrying magnitude of confidence. Such arsenic the bot telling america that “GA4 uses a instrumentality learning-based attack to way events and tin automatically place acquisition events based connected the information it collects.”  

While GA4 does person auto-tracked “enhanced measurement” events, these are mostly defined by listening to elemental codification wrong a webpage’s metadata alternatively than done immoderate instrumentality learning oregon statistical model. Furthermore, acquisition events are surely not wrong the scope of enhanced measurement.

As demonstrated successful our GA4 test, the constricted “knowledge” held wrong ChatGPT makes it an unreliable root of facts. But it remains a precise businesslike assistant, providing archetypal drafts of analyses and codification for an adept to chopped the clip required for tasks. 

It cannot regenerate the relation of a knowledgeable expert who knows the benignant of output they are expecting to see. Instead, clip tin beryllium saved by instructing ChatGPT to nutrient analyses from illustration information without dense programming. From this, you tin get a adjacent approximation successful seconds and instruct ChatGPT to modify its output oregon manipulate it yourself.

For example, we precocious utilized ChatGPT to analyse and optimize a retailer’s buying baskets. We wanted to analyse mean handbasket sizes and recognize the optimal size to connection escaped shipping to customers. This required a regular investigation of the organisation of gross and borderline and an knowing of variance implicit time. 

We instructed ChatGPT to reappraisal however handbasket sizes varied implicit 14 months utilizing a GA4 dataset. We past suggested immoderate archetypal SQL queries for further investigation wrong BigQuery and immoderate information visualization options for the insights it found.

While the options were imperfect, they offered utile areas for further exploration. Our expert adapted the queries from ChatGPT to finalize the output. This reduced the clip for a elder expert moving with inferior enactment to make the output from astir 3 days to 1 day.

Dig deeper: 3 steps to marque AI enactment for you

Automating manual tasks and redeeming time

Another illustration is utilizing it to automate much manual tasks wrong a fixed process, specified arsenic prime assurance checks for a information array oregon a portion of codification that has been produced. This is simply a halfway facet of immoderate project, and flagging discrepancies oregon anomalies tin often beryllium laborious.

However, utilizing ChatGPT to validate a 500+ enactment portion of codification to harvester and process aggregate datasets — ensuring they are error-free — tin beryllium a immense clip saver. In this scenario, what would usually person taken 2 hours for idiosyncratic to manually reappraisal themselves could present beryllium achieved wrong 30 minutes. 

Final QA checks inactive request to beryllium performed by an expert, and the prime of ChatGPT’s output is highly babelike connected the circumstantial parameters you acceptable successful your instructions. However, a task that has precise wide parameters and has nary ambiguity successful the output (the numbers either lucifer oregon don’t) is perfect for generative AI to grip astir of the dense lifting. 

Treat generative AI similar an adjunct alternatively than an expert

The advancement made by ChatGPT successful caller months is remarkable. Simply put, we tin present usage conversational English to petition highly method materials that tin beryllium utilized for the widest scope of tasks crossed programming, connection and visualization.

As we’ve demonstrated above, the outputs from these tools request to beryllium treated with attraction and adept judgement to marque them valuable. A bully usage lawsuit is driving efficiencies successful gathering analyses successful our mundane enactment oregon speeding up lengthy, analyzable tasks that would usually beryllium done manually. We dainty the outputs skeptically and usage our method cognition to hone them into value-adding materials for our clients.

While generative AI, exemplified by ChatGPT, has shown immense imaginable successful revolutionizing assorted aspects of our integer workflows, it is important to attack its applications with a balanced perspective. There are limitations successful accuracy, peculiarly concerning caller updates and nuanced details. 

However, arsenic the exertion matures, the imaginable volition turn for AI to beryllium utilized arsenic a instrumentality to augment our capabilities and thrust efficiencies successful our mundane work. I deliberation we should absorption little connected generative AI replacing the adept and much connected however it tin amended our productivity.


Get MarTech! Daily. Free. In your inbox.


Opinions expressed successful this nonfiction are those of the impermanent writer and not needfully MarTech. Staff authors are listed here.

Read Entire Article