We hear so much about tools, empowerment and productivity, - yet, when an exec pings an analyst to ask a question about a KPI drop, that analyst’s day is basically shot; not just the day, maybe even the rest of the week.
Let’s play out a scenario:
- in the quest for profitability, the operations team launches a new initiative to lower costs
- an exec looking at a key event conversion dashboard notices a meaningful drop around the time of the ops launch
- her interest is piqued, and she pings an analyst immediately - is this driven by the launch?
She’s already hungry to dive deep to understand if specific segments are more or less impacted, what the $ impact is, and is there a balance between the conversion loss and cost savings.
The analyst starts the hunt.
- he wades through the data swamp to find the right table for the conversion metric; there are three similar sounding suspects
- he samples them, compares the outputs, compares to the dashboard; then pings a data engineer on Slack to get confirmation
By now, an hour has elapsed, and he hasn’t even started the analysis.
- he needs dates for the launch, and sends another Slack ping to someone in operations
- he pulls the metric for 2 weeks before the launch
- he finds an unusual drop in one of the 2 weeks prior - so, he has to further drill down, ask someone what happened, then remove specific days to create a “normal” baseline
Then, he repeats this whole process to bring in the operational metric impacted - and by joining against conversion, he can finally analyze the ops change vs the conversion change. He informs the exec that it is “likely” that the operational initiative is affecting conversion.
The exec immediately asks to dig in - does this hold for everyone? is the drop similar, worse or better for various sub-segments?
There maybe 25 valuable attributes adding up to 400 distinct sub-segments to loop through but
- the analyst has no visibility to many of these; it’s all over the swamp
- he picks the most common 2 attributes and 10 sub-segments
He turns around this analysis but the exec is persistent - she wants to dig into more.
We are already in day 2. And as this continues, it will likely last the full day and we haven’t even reached the most important question: can the ops team make changes which could achieve an optimal state trading-off conversion for cost savings?
I played this scenario out in detail to make two points
- analysts are unsung heroes behind the scenes
- we are still very early in the journey to improve tooling and productivity for data consumers so these types of workflows can be executed in a few minutes aided by powerful software
In this tight economy, we can’t do more with less just by saying so. We need the tooling to support it.
