Data-Driven Product Roadmaps: Choosing the Right Metrics

Jim Semick
Co-founder, Board Member at ProductPlan

Data Driven Roadmap

Developing an effective product roadmap requires more than a product manager’s intuition and vision, although those elements are crucial. But equally important for an effective roadmap will be evidence, real-world data, and a scientific approach to analyzing that data.

In this post, I’d like to share with you some ideas for identifying, tracking, and leveraging the right metrics to help shape your product strategy. These are some of the ideas that I shared in a recent webinar that we co-hosted with Pendo, called “Building Data-Driven Product Roadmaps”.

Not All Data Are Equally Valuable (Beware of the Vanity Metric)

Before we discuss how to leverage data in prioritizing your product roadmap, I want to caution you upfront that not all data is equally useful. Some data, in fact, is a distraction at best, and a source of misleading information at worst. In particular, I want to warn you against focusing too heavily on “vanity metrics.”

Vanity metrics are stats that look great in a press release but don’t necessarily translate in any meaningful way to business results. Examples include the number of Twitter followers or how many people have watched a clever video your company posted on YouTube. In reality, these numbers give product management very little insight into how the product will resonate with customers or how much revenue it will generate.

Tweet This:
“Not all data are equally valuable (beware of the vanity metric).”

Worse, when you rely on these sorts of metrics — which can be tempting because they suggest interest and engagement with your company — you risk developing a very skewed view of the actual level of interest in your products. People watch YouTube videos and follow companies on Twitter for lots of reasons, many of which have nothing to do with an interest in those companies’ products or services.
Download Product Success Metrics  ➜

Limit and Prioritize the Metrics You Use in Your Roadmap Decisions

Another important point to keep in mind is that with services like Google Analytics, it’s possible to track many different types of data. And that’s great.

But trying to make sense of an overwhelming amount of data can also lead to confusion. You probably have a firehose of information available to you right now, so you should first try to figure out what the most important metrics for your product are. This includes the data that will lead to the most valuable and actionable insights into what’s working with your product, what isn’t resonating with users, and how you can continually improve your offering.

The ideal number of metrics to track will vary from company to company, of course, but a good rule of thumb is to focus on roughly five metrics that you can monitor over time and discuss regularly with your executive stakeholders.

Assuming you’ve identified the five most revealing and strategically advantageous metrics to concentrate on, the data you compile from these metrics over time should give you valuable intelligence into how to proceed with your product and what to prioritize on your roadmap.

First, you have to set your product’s strategic goals. Before you can begin sifting through all of the noise and determine which handful of strategically useful metrics to focus on, you need to define your product’s strategic goals.

Then, when you’ve set those goals, you can start looking for the right mix of metrics to track and analyze, the metrics that will help you achieve those goals.

Next, start compiling data — here are some great places to look. If you already have a product on the market, with an installed base of users, you can choose from a wide range of real-world metrics to track, which will help you take a more scientific approach to decide where to focus your resources.
Get the Product Roadmap Kit ➜

Customer-Oriented Metrics

A great way to start is to use the data being generated every day by your prospects and user base to learn what’s working with your product or your efforts to promote it, and what you need to change. These types of metrics might include:

Business-Oriented Metrics

You can also track and analyze important high-level business data relating to your product to help you shape and improve your strategy. These business-oriented metrics will come not from how your customers interact with your product but rather how their actions affect your product’s bottom line. These types of metrics could include:

To use my own company as an example, at ProductPlan we pay close attention to a handful of these metrics all the time, and they are extremely helpful in guiding how we pursue and adjust our strategy.

One of the great things about offering a SaaS-based product, which we do with our product roadmap software, is that because you are continually tracking everything in the cloud, you can view and analyze an incredible amount of detail about how your customers are interacting with your product. Even though we don’t have access to our customer’s roadmaps for security and privacy reasons, we are aware, for example, of how many product managers using our software to share their roadmaps with others across their organizations.

This helps us to determine whether our tool is encouraging communication across the company and whether there are things we need to do to make the process easier or more intuitive for our customers. And if there is a common point in the roadmap development process where customers tend to reduce their interaction with our product, that gives a signal that we might need to revisit that part of the user experience and streamline or refine it.

Where to Find Useful Metrics for Your Roadmap Before You Have a Product

The examples of metrics that I listed above work well when you already have a product out there, generating data that you can track. But what if you don’t yet have a product on the market? What if you are in the strategy stage, just preparing to build your product? Where can you look for the metrics that will inform your strategic decisions? Here are three useful sources of information:

Experts in Your Industry

These could include industry experts, or the widely read columnists and commentators who write and speak about trends in your industry and review your competitors’ products.

Industry Analysts

The researchers who cover your industry are an excellent source of information about which metrics offer the most significant clues as to what will constitute a successful product. These are the people, after all, who spend their days listening to briefings from your competitors about the things they’re finding resonate with their customers, and who then write detailed reports about which products are enjoying the most success in your industry, and why.

Your Competitors

Unless you’re planning to create an entirely new category with your product, chances are you have competitors with products already on the market. And chances are those competitors have done a lot of the research into the most strategically useful metrics relating to their products and their customers, and may have public information available, especially if they are a publicly-traded company. You can leverage this information in developing your own set of metrics to focus on.

You can find your competitors’ discussions of what they deem important data points in their press releases, on their earnings call with analysts (assuming they’re publicly traded) and even in their own business filings online.
Download Product Success Metrics  ➜

How to Communicate Your Metrics-Driven Strategic Goals

To this point, I’ve been discussing the importance of using a small set of key metrics as a way to help you shape your product strategy.

But another important reason to track and analyze these metrics — equally important in helping you successfully bring your product to market — is that they will give you a set of concrete evidence to share with your executive stakeholders over time, to help them understand and support your strategic decisions for your product.

One of the most effective ways to communicate your product strategy with your stakeholders, and to share with them the evidence that supports your strategy, is with a visual, intuitive product roadmap.

As you can see from the screenshot below, depicting a roadmap developed using ProductPlan, when you use the right roadmapping tool you can visually articulate your product’s strategic goals and then build into the roadmap the data points that support your decision to pursue each of those goals.

Product Roadmap Template

Conclusion: The 6 Steps to Data-Driven Roadmaps

So, to recap, my advice would be to always build your product strategy, and to develop your roadmap, based on key metrics that support your strategic decisions. And to accomplish this I suggest the following steps:

1. First, establish your product’s strategic goals. (You can always adjust these goals in light of the evidence you’ll be gathering.)

2. Next, make a list of possible strategically advantageous and illuminating data points to track — such as customer-oriented metrics like product usage, and business-focused metrics like customer acquisition cost.

3. If you don’t have a product already on the market (or even if you do), you might also want to research your industry for strategically important metrics to focus on. To find these metrics, check in on what your industry’s luminaries, media, analysts, and competitors are focused on.

4. Now it’s time to limit your metrics to a handful or so that you determine will give you the most strategic insights for the time and energy you’ll invest in tracking and analyzing them. (Too many data points can lead to confusion and can dilute whatever strategic insights you might uncover.)

Tweet This:
“Tie metrics to strategic goals, and if the data shows you need to adjust your strategy, do it.”

5. Next, tie these metrics to specific strategic goals you established in step 1. (And if your data points lead you to adjust your strategy, now is the time to do that.)

6. Now you can build this strategy (including the data points supporting it) into your product roadmap, which you can then share with your executive stakeholders — demonstrating to them that you’ve brought evidence, and not just your intuition, to these strategic decisions.

At the point, I’d typically wish you luck with your product’s success. But now you’ve got evidence backing your strategy — so you won’t need luck.

ProductPlan Email Course on Metrics and Data - Register for Free