Going Overboard on a Data-Driven Approach (How to Avoid the Trap)
These days, product management and a data-driven approach are unshakeably tied together. Most want decisions based on a solid foundation of facts with experiments...
Key Performance Indicators are the lights on a foggy landing strip, a beacon on the horizon of a choppy ocean, and a GPS as you try finding your way to an unfamiliar destination. Pay attention to them, and your actions will stay aligned with your strategy. Ignore them and increase the odds of an unpleasant outcome.
But like any metric, slavishly pursuing a higher grade while ignoring everything else is also a risk. KPIs are necessary, useful tools, but so are common sense and general business savvy.
To make the most of KPIs, it’s important to select them wisely. Monitor (but not obsess over) them and ensure they’re still the most accurate measures of the current strategy. But before going any further, let’s make sure everyone understands what KPIs are and what they are not.
A KPI is a quantitative metric that organizations use to track progress toward business goals. In plain English, it’s a way to measure the things that matter most.
KPIs add value because many things could be measured and tracked when data is constantly streaming in, and analytics packages are readily available. KPIs narrow things down and highlight the metrics that matter while shoving vanity metrics to the sidelines.
For example, signups for a service could be viewed as an excellent way to track success. But if a large percentage of those users abandon the product, that metric can be misleading. It may signal that the marketing message or target is off, or that the product isn’t meeting expectations. In this case, a company might decide a better metric is churn rate and—assuming customer retention is critical to the corporate strategy—make THAT the KPI versus new users.
Of course, a KPI isn’t just a preferred metric; it’s also associated with a goal. So in the example above, it’s not just tracking the churn rate, but quantifying how the team is reducing it since the business goal is increasing the number of engaged and loyal customers.
The alignment with strategic objectives is also important because it builds on what stakeholders already care about. Their interest in a KPI and support for any product decisions that improve it directly links to the overall product or organizational targets.
It isn’t just a battle of three-letter acronyms. There’s an essential distinction between these two types of guideposts for products and businesses.
OKRs (Objectives and Key Results) aren’t quite the same thing as a KPI. While a KPI measures performance for an established product and a defined, achievable goal, OKRs are more often established for brand new goals or initiatives.
It is mostly due to the “O” in OKRs, which includes the actual objective setting itself (and not just seeing how things are progressing). OKRs are more common when the organization is trying to do something new or make a strategic change.
When a new objective is defined (i.e., entering a new vertical), its key results are what the team expects to happen based on this decision and the subsequent execution. These key results are more like milestones indicating that you’ve reached the objective more than ongoing measures of progress or sustained success.
So in this example, the key results might be that they added industry-specific functionality to the product, and three customers from that industry have signed contracts or are using the product. A KPI, in this case, might be looking for sustained growth in the new vertical, such as a percentage of business coming from that industry or utilization of those new industry-specific features.
Although these are subtle distinctions, they do make a difference.
KPIs are the “key” metrics you track for a particular product or the overall business. There can only be so many measures pivotal enough to have that distinction. Therefore only a select few can cut it.
One tactic is identifying the One Metric That Matters (OMTM). Suppose you could only improve one particular thing, which would have the maximum benefit for the overarching product strategy. This single metric becomes the standard-bearer, with all other KPIs taking a backseat in terms of priority.
With the OMTM set, the team can come up with two or three complementary KPIs. These might be selected to address the concerns of key stakeholders or because they address secondary goals.
If you’ve chosen too many KPIs, there’s a lack of focus and potential conflict. Remember, the goal of KPIs isn’t to determine what to measure, but what measurement is a priority.
You can still track the rest of the metrics, and some might eventually turn into KPIs themselves if they’re found to be worthy leading indicators or current business goals change and evolve. For example, during a product’s early days, revenue is seldom a KPI, but gross sales or profitability may rise in strategic importance.
The resources working on a product—from development to marketing to sales—will be rightfully focused on improving KPIs. So if there’s any disconnect at all between those measures and the product strategy itself, it’s likely that the KPIs will lead those well-meaning employees astray.
Since a product strategy isn’t in itself measurable, the KPIs will garner the attention and guide the activities of the staff. It’s not reasonable to expect them to parse out whether the KPIs are still accurate, relevant, and supportive of the product strategy, notably if it changed in any way.
Stakeholders must ensure KPIs truly represent the goals of the organization, accurately matching the latest version of the product strategy. Therefore it’s essential to periodically review KPIs not just for their progress but also their fit with the current line of thinking.
There’s no shame in switching up KPIs now and then to fit better what the business or product is currently shooting for. An occasional refresh of tweak will inspire more confidence in the team that what you’re measuring and prioritizing is supporting the latest product strategy.
KPIs intend to provide laser focus on a few important “needle movers” that the organization values. Therefore, the product roadmap shouldn’t prioritize things that don’t directly or indirectly improve those KPIs.
It provides a highly effective filter for prioritization activities. You can judge every potential enhancement, project, or theme by whether or not it will positively influence one or more KPIs. The greater the impact, the more important the initiative, since improving KPIs is the best way to implement the company strategy.
KPIs also empower product managers to say “no” with authority. Product managers can quickly shelve all those random requests from customers, salespeople, and executives that don’t help with a KPI. While that’s sometimes easier said than done, if the organization values KPIs enough to institute them, they need to walk the walk when it comes to resource allocation.
Metrics and planning are a two-way street. For short-term, day-to-day decisions, KPIs are an excellent way to validate or drive planning activities since the focus and motivation center on improving those metrics.
It doesn’t mean robots can replace product managers just yet. There is still plenty of judgment and thoughtfulness required to determine IF a particular decision will have a positive effect and just how significant that might be. But it does provide a constant framework for how to evaluate things. It includes near-term product roadmapping, as its themes, initiatives, and milestones should be aligned with improving the current KPIs.
When it comes to annual planning, however, things should work the other way around. Major strategic planning should be determining the major goals for the product and business, not metrics.
Only after setting those ideals should you begin reviewing the fit of existing KPIs and identifying better ones. Any strategic adjustment should always include a reassessment of current KPIs to ensure they’re still a match for the plan.
Data-driven decision-making isn’t a fad. KPIs are management tools that align the entire organization to focus on what’s most important. Product managers must understand the full implications of KPIs and how they’ll influence much of what they and their colleagues do daily.
But metrics are only valuable when they measure things that matter. Product managers have a critical role to play in ensuring that’s the case. Learn more about how metrics impact product management and how to make use of the best. Check out our free resources on this topic: