Magoosh, an online test prep company based in Berkeley, California, calls Net Promoter Score their “reliable referral indicator.” And it is. But that’s not all it is. Although many people think of NPS as a customer success or support metric, product developers in innovative companies, like Magoosh and so many others, are using it to help guide their product roadmaps.
What does NPS guide them towards? Aligning product around users’ desired outcomes.
“At Magoosh, NPS is one of the most important metrics we track – it helps us determine not only whether students like our customer service and user interface, but also how well our products prepare students for their exams.” – Peter Poer, Director – Product Content
How Product Teams are Using NPS to Build Better Products & Better Experiences
We often see product teams using NPS to track customer happiness (it’s a lean way to get a constant stream of customer sentiment). But, it’s also incredibly useful for helping to prioritize improvements and allocate resources. As Jason Lemkin of SaaStr says – “it is the one metric that keeps SaaS companies honest.”
Let’s say your NPS dips. Now what?
When NPS alerts you that there is a problem, the qualitative feedback that NPS responders offer can often tell you what the issues are. But, figuring out exactly what went wrong is one thing – finding the best way to solve it for customer happiness is another challenge altogether.
Use A/B Testing with NPS
This is where Magoosh, a test prep company, brought in A/B testing. They’d identified a falling-off point for their scores after students took the actual GMAT test. Looking into why, they found that passive and detractor students complained that their actual scores were lower than their practice test scores.
“Our algorithm was telling students to expect one score, but for some, their official reports were coming back lower – obviously a frustrating experience.”
The solution was to fix their score prediction algorithm to be more accurate – but they hesitated: Would it be more frustrating for students to get lower scores on their practice tests, or disappointing scores on the actual tests?
Their approach to solving this problem was to use A/B testing to, essentially, optimize for NPS.
They deployed the updated algorithm to half of the GMAT students and kept the other half on the previous algorithm. They were able to ask the NPS question and get feedback from currently studying students in real-time, instead of waiting to ask months later after students had taken take their exams and received their results.
“Suddenly, NPS had a new use case for us – as a powerful, agile product tool.”
Magoosh discovered that the more accurate algorithm had no effect on student satisfaction while studying with Magoosh, which allowed them to roll out the change to all students more quickly.
The results: Students from the A/B test who studied with the updated algorithm had their NPS jump 9 points after they completed their exams.
Magoosh used NPS data as an agile tool to shape the evolution of their product, prioritize updates, and act as a compass for their product roadmap.
Effectively Use Qualitative Data for Product Dev
The NPS survey asks for qualitative feedback – the “why” behind the number. But there are right ways and wrong ways to use the ‘why’ data you get. And, if not used correctly, you can waste a lot of your product team’s time chasing the wrong leads.
Segment by Promoter, Passive, Detractor
First, you’ve got to understand where your “why”s are coming from. Promoters? Passives? Detractors?
Promoters will tell you what you’re getting right, and maybe, sometimes, what they’d like to see improved. Take their feedback very seriously, because they are your best customers, your target audience, the whole reason you’re sitting in your chair, gainfully employed.
Passives and Detractors can be a mixed bag, and here’s why: Some of these Passives and Detractors are not your ideal customers, or target clients. Some of them are just in the wrong place, using the wrong product for their needs. Be cautious about taking their “why”s as directives, you may make your product a worse fit for your Promoters and waste time and money doing so.
However, most Passives and Detractors are people who want to love your product. And they graciously contributed their time to telling you what you can do to make their experiences better.
Be sure to follow-up with them and thank them all, because they’re doing you a big favor.
Segment responses by key user properties
Different categories of users have different needs and will experience your company in slightly different ways. You can segment NPS by any property that you have on your users – think about your business drivers. An Enterprise user may have a higher LTV than a Pro plan user. Allocate resources accordingly.
Track themes in the feedback
Keep score of trends in your feedback, and once you’ve identified a problem experienced by many, A/B test your solution.
It is a bit easier to filter and analyze trends with tagging, which gives some structure to qualitative feedback. You can use tags to track common themes related to your customer’s experience of your product or app — such as features, usability, support, pricing, performance.
Conduct Additional Research with NPS
NPS can help product development teams most when supported with deeper research, and deeper research can be facilitated by NPS segmentation. Hubspot’s product managers, for example, reach out to select NPS survey respondents to investigate further.. In one instance, they found that a mobile app they had created wasn’t getting as many recommendations as they’d hoped, so Hubspot reached out to a subset of users who had a high NPS for the desktop version, but a low score for the app version, to locate the disconnect.
Their top tip for reaching out to people is a good one: “Make it clear this request is coming from the product team.”
Customers are contacted by a lot of departments – sales, customer service, marketing. But when a call for help comes from the people making the product, it sends a strong message that your company is deeply invested in their success. You’ll get a higher response rate, and very likely, rich feedback.
And Don’t Forget to Share
Net Promoter Score isn’t just a “customer service” thing, or even a “product development” tool – it’s a tool every team can use to do what they do better. But, sometimes, data stays stuck within the product team, or within customer service. Don’t be afraid to take the lead and close the loop! Pass that data around to other teams. Pass promoters to marketing or sales. You have a wealth of data at your fingertips – don’t forget to share.
About the Author
Jessica Pfeifer is Co-founder and Chief Customer Officer at Wootric. Jessica has onboarded and advised more than 200 companies on effective use of the Net Promoter System. Prior to co-founding Wootric, Jessica spent six years growing brands at The Clorox Company. She also spent five years in China leading customer experience management initiatives for Aon Hewitt. Jessica holds a BA from Yale University and an MBA from Harvard Business School.