While data producers—like data engineers—have benefitted significantly from applying engineering principles to their work, data consumers have largely been left behind in the process.
Data teams have been building infrastructure and creating sophisticated pipelines and data models to make data reliable and accessible. But for the vast majority of users, even with these technical advancements, data consumption still remains a fragmented and inefficient experience.
We need to apply a different set of engineering principles designed specifically for data consumers. Here’s some ideas for how we can better empower data consumers.
1) No-Code Abstractions
The first principle for data consumers is the need for no-code abstractions. If we're still exposing unnamed datasets and reams of sql code, it's near impossible for business users to consistently derive value from their data.
Metrics and metric relationships must become the new units of abstraction for business users. By utilizing metric trees, we can structure data into comprehensible abstractions that aligns with how the business works and how the business users think and act.
2) Out-of-the-Box Calculations and Algorithms
But simply having structured data isn't enough. We also need out-of-the-box calculations and algorithms to support common questions and workflows for end users. Data consumers often have to resort to manual computations or spend excessive time trying to understand complex datasets to answer repetitive business questions.
To address this, we must provide ready-made calculations and algorithms that are tailored to common business needs like revenue tracking, retention analysis, or customer segmentation.
3) Clear Definition of Common Workflows
Even with no-code abstractions and built-in calculations, the process won't work unless we clearly define common workflows that users follow. Business reviews, metric root-cause analysis, variance analysis against budgets, pacing metrics—these are all common workflows that data consumers engage in.
By defining the scope of these workflows, and tools built with these in mind, we will make it significantly easier for users to quickly jump into their work without struggling to piece together these workflows from scratch every time.
4) Workflow Automation
Finally, to truly unleash the power of data for consumers, we need to integrate workflow automation into the process.
The goal should be for the software to do the work proactively. Imagine systems that automatically flag anomalies, surface insights, and highlight areas where actions are needed.
In summary, if we are to truly empower data consumers across an organization, we must rethink how we design data systems and workflows. By focusing on no-code abstractions, providing out-of-the-box calculations, defining common workflows, and automating these processes, we will transform data consumption from a frustrating, fragmented experience into a streamlined, actionable and delightful experience that drives better business decisions at every level.
