The Brutal Truth About "Data-Driven" Organizations
Most companies claim to be data-driven, but after two decades working with organizations from startups to Fortune 500s, I’ve discovered an uncomfortable truth: The majority of the metrics these companies track aren’t usable or used ineffectively to make product decisions.
You may have experienced this yourself: Your company is collecting an overwhelming amount of data, but none of it provides insight into the relationship between the business results you’re getting and the work your teams are doing. It might look impressive on paper, but is it really telling you what’s actually happening in the business? Does every product decision need to be metric-driven?
In other words, is the data you’re using serving any meaningful purpose?
Odds are, it’s not. And here’s why.
The Disconnect Between Metrics and Impact
One of the biggest challenges organizations run into at the product level is being unable to effectively connect product features to business impact. (I’ve written about this in my last post—check it out here if you missed it!)
The misconception here is that more data-driven practices will solve the problem. But even when companies diligently track metrics, they're often measuring things that don't actually move the needle. Why? Because:
- There's no clear understanding of how individual metrics influence company goals.
- Most metrics are operational vanity metrics rather than true business health indicators.
Conventional wisdom has long been that “What’s not measured isn’t actionable.” This has led organizations to track dozens—sometimes hundreds—of metrics. Incentivized by this, many teams pride themselves on measuring extensive metrics to demonstrate how comprehensive their systems are. But this approach actually undermines data effectiveness through information overload.
Product leaders often feel this acutely. You could be drowning in numbers across 30 different dashboards and still be unable to extract any actionable insights. Or worse still, the insights you extract directly contradict each other, leading to decision paralysis.
Transforming Your Approach
The solution to this problem isn’t to do away with metrics altogether, but to prioritize focused measurement of critical markers that tie directly to business outcomes. When you reduce your KPIs from, say, 27 to 3, it becomes a lot easier to tell what moves the business. This approach delivers far more value than comprehensive measurement of everything possible.
But there’s more to streamlining your metrics than just arbitrarily reducing your KPIs. Here are five additional strategies for making your data useful, not just comprehensive.
1. Data Only Tells Part of the Story.
Not every product decision should be metrics-driven. The problem with being driven solely by metrics is that we rarely have all the metrics we need to make “data-driven” decisions. Nothing can beat empathetic understanding of space. Also, true innovation requires more than a metrics-based approach.
Remember, quantitative data is good for optimization situations. Qualitative data is excellent for net-new innovation or first principles ways to address users' jobs to be done.
2. Integrate Product and Business Teams
As I touched on in my last article, product teams tend to become detached from business realities, while platform and infrastructure teams are even further removed. This fragmentation creates silos that prevent meaningful impact.
The solution is to create integrated teams where product, engineering, and business stakeholders work together to understand business drivers and decide which outcomes are most meaningful and most worth measuring.
3. Ensure Metrics Are Measurable and Actionable
I’ve seen countless teams propose elegant-sounding metrics that are impossible to measure consistently. Take “productivity,” for example. How do you define it? How do you measure it accurately? It’s vague, and so imprecise that you’d be better off focusing your attention elsewhere.
So what makes an effective metric, anyway? The criteria include:
- A clear baseline
- Consistent measurement methodology
- Monthly tracking capability
- Communication that explains movement and reasoning
Understanding why a metric is useful, and why it changes, is absolutely essential if you want to extract any useful insights.
4. Invest in Data Infrastructure
Many companies fail to act on data because they lack modern tools and infrastructure. The old approach of Excel projects and siloed documentation is insufficient for today's complexity. If you want your organization to be truly data-informed, you have to invest in infrastructure and tools that make metrics widely accessible.
AI-led organizations employ cloud-based tools that provide real-time visibility across the business, enable programmatic workflows, facilitate collaboration, and serve as a canonical repository of completed work. However, you have to walk before you can run. Today’s organizations often attempt to layer sophisticated analytics and AI tools on top of inadequate data infrastructure. This approach inevitably fails.
It doesn’t matter what AI tools you deploy or how you restructure your teams. If you don't have strong architecture to support your data needs and high-quality data feeding your systems, your efforts are likely to fall short of expectations.
5. Rethink Organizational Structure and Culture
Traditionally, data teams have been centralized, serving the entire organization. To become truly data-led, you should be considering a hybrid hub-and-spoke structure: a central team handling organization-wide processes with smaller pods supporting specific business units. This allows you to devote specialized attention where it matters most.
Organizations also need to be rethinking their culture around data-sharing. Metrics are how we collectively understand business performance, and they aren’t meant for one single team. When teams establish a culture of sharing data across departments, the entire organization gains a better understanding of where to invest resources.
The Path Forward
As data proficiency becomes table stakes for most organizations, your data capabilities can be a valuable competitive moat… if used right. Business leaders and product organizations need to be embracing a model that supports growth and accommodates increasing complexity, but contrary to popular belief, “more” isn’t always “better.”
The organizations that gather and use data the right way focus first on building reliable data infrastructure, then on measuring and acting on what matters. If your data practices are outdated or overwhelming, now is the time to shift toward a more effective approach. And by applying the strategies in this article, you’ll be off to an excellent start.