Is the mere existence of data analysts an anti-pattern? And if so, does the rise of AI threaten their existence?
Looking at this through the lens of jobs-to-be-done in the data space, we can break down the work into three key roles:
- The Data Producer: Typically an engineer, responsible for building the platform that ingests data and creates useful data models.
- The Data Consumer: This can be anyone from a data scientist to a growth marketer, product manager, or FP&A analyst. They consume datasets and further transform them for business use cases, such as deep dive analysis.
- The Decision Maker: This is the business stakeholder interpreting the analysis, drawing conclusions, running experiments, and shaping strategy to influence the key levers of the business.
In this pipeline, the role of the data consumer (or analyst) is essentially an intermediary. Their responsibilities include:
- A deep understanding of the datasets and the business context in which they are collected.
- The ability to transform data directly using code or in spreadsheets for various use cases.
- Applying their numeracy skills to execute analysis, interpret results, and recommend actionable next steps.
Now, to answer the first question: Is the role of the data analyst an anti-pattern?
Intermediary roles exist across almost every function. These roles arise to bridge clear gaps in expertise or process, which are present on either side of a job-to-be-done. So no, data analysts are no more an anti-pattern than, say, engineers who code ideas for marketers.
They’re an essential bridge between the data producers and the decision-makers, interpreting and translating complex data for business needs.
However, AI does pose a threat to this intermediary function? AI can accelerate workflows for about 60-70% of use cases by automating tasks like building reports on well-defined data models executing routine data transformations.
But AI will struggle with more nuanced data models, complex concepts, and deeper analyses that require higher human cognition, intuition or domain-specific knowledge.
The obvious consequence of AI’s rise is that data teams will become smaller and more nimble, with fewer analysts within organizations. This is inevitable as AI handles more routine, transactional tasks.
But there's an interesting counterpoint: Unlike sales or engineering teams, which can easily grow and remain bloated due to the belief in their ability to influence business outcomes, data and finance teams rarely bloat. In fact, market forces already naturally right-size these functions.
In summary, I believe data teams have already corrected themselves and are operating lean. With the rise of AI, I expect them to remain lean—and possibly even leaner.
However, it is unlikely that AI will fully replace the core analyst function anytime soon. Analysts are needed to handle the more complex, nuanced aspects of data, and AI is still far from being able to execute those tasks at the same level.
