Building and maintaining the product roadmap is a central part of your role as a product manager. Yet there is surprisingly little consensus about product roadmaps across the product management community. Opinions vary wildly, for example, about what exactly a product roadmap is, how to structure one, what to include in it, and which tools you should use to develop it.

In this post I will offer a few guidelines for how to structure your product roadmap in ways that can lead to the development of a successful product. But before we dive into these suggestions, I would like to start with two fundamental points about roadmaps — points I hope will make the guidelines that follow much clearer.

The top-down approach to product development works best.

For successful product development, I recommend a top-down product strategy. Product roadmaps fit very strategically into this hierarchy. Here’s how it works.

Start with your product’s vision — which you’ll derive from your company’s larger strategic objectives. Then translate that high-level vision into actionable goals. Next, turn those product goals into your product roadmap. Finally, move from your roadmap to your backlog.

Starting at the highest level, and working your way strategically down into the details, is the best way to stay focused and on track toward your main objectives — and to avoid losing sight of why you’re doing what you’re doing and getting lost in the weeds.

And speaking of getting lost in the weeds…

A product roadmap is not a list of features.

A list of features is just that — a list of features. The product roadmap, on the other hand, is a strategic document that represents your high-level goals for the initiative along with an execution strategy to communicate how you plan to achieve those goals.

A roadmap might include features, of course, but the features themselves are only a part of the execution plan.

Five Guidelines for Structuring Your Product Roadmap

1. Product Roadmaps Should be Flexible

You’ve got to be okay with uncertainty. You’ve got to be willing to embark on your product plan, knowing that you can’t possibly know everything at the outset — that you will hit surprises, challenges, and opportunities along the way. Your roadmap needs to factor in these uncertainties — and needs to be flexible enough to allow you to change course quickly when necessary.

To avoid making promises you can’t keep, make sure your stakeholders understand that the roadmap is not an end-all, be-all document. One way to do so is to show more granularity in the short term while keeping your initiatives high-level and your dates approximate in the long term.

Thinking in themes is another effective way to keep your roadmap flexible. For example, maybe you and your stakeholders have agreed to focus on increasing conversions from a particular target persona in the upcoming quarter. This area of focus becomes your theme, and you retain the ability to reprioritize specific features and fixes within that theme while keeping the overarching goal intact. Likewise, if an opportunity arises that was not originally on your roadmap but furthers your shared goal, you’ll have an easier time making a case to include it.

Finally, a flexible roadmap can be a headache if changes are not communicated promptly and effectively. Ensure your stakeholders always have access to the most up-to-date version of the roadmap, whether that means storing it on a shared wiki or using cloud-based software that automatically updates. Transparency is key to building alignment on product strategy, no matter how frequently you need to reprioritize.

2. Product Roadmaps Should be Deeply Rooted in Well-Thought-Out Goals

This might sound like a contradiction to the previous guideline — but it’s not. You can frequently reprioritize your backlog while still staying true to well-thought-out, agreed-upon goals.

Your specific roadmap priorities might need adjusting in light of new information — in fact, they almost certainly will at one time or another. But you should be prepared to adjust focus in your roadmap, reallocate resources or otherwise change direction only if doing so is in alignment with your product’s high-level strategic goals and vision.

As a product manager, it’s your job to make sure that you’re making decisions about what to do, and what not to do, for the right reasons. Even if a sales executive is loudly demanding the immediate inclusion of a new feature to close a deal, you still need to weigh the request in light of your organization’s strategic goals. Quick wins do not necessarily spell long term success. If the feature isn’t in line with your product vision, it’s okay to say no. Indeed, if you’re doing your job well, you’ll find yourself saying no quite often.

3. Product Roadmaps Should be Developed with Plenty of Input

You don’t need to craft your roadmap yourself. You shouldn’t. Silos rarely work for any business function, and for roadmaps they can lead to ill-advised plans and products developed without critical knowledge.

To successfully bring a product to market, or to update an existing product in such a way that benefits the customer and your company, you’ll need plenty of input from experts across a variety of teams and departments. That includes, for example, engineering, customer support, sales, and marketing.

Sales, for example, will have valuable anecdotes about their most recent wins, or their most important ones. They can also tell you about the features prospects and customers are asking for.

As for your engineers, the more you involve them in the creation process, the more ownership and responsibility they will take for their role, and the more creativity and enthusiasm they’ll bring to the project. Collaborating with your engineers, and soliciting their help, will make them feel more like a part of the process, and less like order takers simply being told what to do by a product manager.

Prabhat Jha, CTO of the Net Promoter Score platform InMoment adds, “Be sure to solicit plenty of user input. Some years ago I learned this the hard way. The software enterprise where I worked did not have a rigorous system in place for listening to customers. We had collaborated internally on our roadmap priorities and thought we knew our customer’s needs. We missed key features, though, and as a result, there was very little adoption of the first version of the product. We ended up having to do a second release very quickly in order to get traction.”

4. Product Roadmaps Should be Visual

Ultimately, a product roadmap is a communication tool — an execution strategy you will use to convey your product plans and goals to a variety of constituencies. And the best way to communicate a complex initiative is visual. If your roadmap is simply a long list of features presented in a spreadsheet, people’s eyes will glaze over.

Visual roadmaps make it easy for everyone in your strategy meetings to quickly understand what you are proposing and hoping to accomplish quickly.

Another valuable reason to make your product roadmap visual is that it forces you to be ruthless about which initiatives to include and which to leave out. Venture capitalist Guy Kawasaki’s “10/20/30 Rule of PowerPoint” instructs that PowerPoint presentations should have 10 slides, last no longer than 20 minutes, and contain onscreen text no smaller than 30 points. In addition to making the presentation more digestible for an audience, this exercise forces the presenter to distill both the whole talk and each slide down to its most essential elements.

The same is true for product roadmaps. If you use a visual tool for creating your roadmaps — such as PowerPoint or a visual roadmap software application — that exercise will force you to distill your plan down to only those initiatives that serve your product’s strategic goals and vision.

Looking at this from another angle, if your roadmap contains 1,000 initiatives, that probably means you haven’t done a good job of prioritizing and strategizing what needs to be included in your product. Indeed, if you do have a list of 1,000 initiatives, your document could probably be more accurately called a backlog than a roadmap.

5. Tailor Your Roadmap Discussion to Your Audience

You are likely going to show your product roadmap to many different constituencies, in many different meetings, throughout the development cycle of your product. Each constituent group will have a unique focus and set of priorities, and each meeting will call for you to delve into different aspects of the product roadmap.

There are a couple of ways of accomplishing this. You can create separate roadmap documents for each constituency, or you can use a single, general roadmap and zoom in to what’s important for each group. The important thing is that you provide your stakeholders context and show them how your plans will further their unique goals.

For example, for your sales team, you might want to highlight the aspects of your roadmap that are designed to bring them better-qualified leads — for example, a trial version of your software that can help to prequalify interested prospects.

Customer-facing teams also likely had input into the features that made it onto your roadmap in the first place. Be sure to communicate to them where their requests fit into the plan (or didn’t) and why.

However, when presenting the roadmap to your executives, you’ll want to focus on how the choices you’ve made will lead to increased revenue or grow market share.

And when meeting with your engineering team, you’ll want to focus both on the high-level themes and specific feature details and discuss how your engineers can help to make those product goals a reality.

In other words, you want to keep your product roadmap flexible enough that it can help facilitate a productive meeting at any level of detail needed, with any constituency group you are presenting to.  

Conclusion: Whatever structure your product roadmap takes, its main job is to communicate your strategy.

A product roadmap needs to communicate your strategy. It is your job to create your roadmap in such a way that it lays out a high-level execution plan for the product’s successful development and eventual launch into the market. It’s also your job to make sure all relevant constituencies understand the goals of the roadmap — and work with you to achieve them.

New things will always come up — cool ideas for new features, requests from executives for a shift in priority,  urgent demands from sales reps, and so on. The challenge for the product manager is to view each one as an opportunity to evaluate through the lens of strategy and goals and help drive a sound decision-making process.

That’s why you need to start with your product vision, and from there derive specific goals — and only after you’ve developed those goals, build your product roadmap. If you haven’t fully fleshed out your vision and strategic goals for the product, it’s too early to start building your roadmap.

About the author:
Andre Theus is the Vice President of Marketing at ProductPlan. He works closely with customers and prospects to build better product roadmap software. Prior to ProductPlan, he was a member of marketing teams at RightScale, Sonos, and Citrix. Andre received a master’s in computer science from the Cologne University of Applied Science in Germany.

In this guest post, Nathan Lippi, Head of User Research at PandaDoc, shares a Pareto principle approach to getting the most from a B2B Net Promoter Score program. 

NPS. It’s debated, loved, and hated, but in the world of B2B SaaS it’s rarely used to its full potential.

At PandaDoc, we’ve become increasingly customer-obsessed since the introduction of our NPS program two years ago, but we feel as if we still have meaningful room for improvement.

We’ve found there isn’t much written about NPS, specifically for B2B companies, so in order to level up, we’ve gone straight to the experts. With their permission, we’re sharing some key findings here.

We hope this guide helps you to get the most out of your CX program.

Let’s get to it!

The main purpose of NPS is to drive action

NPS is an easy, trusted, and benchmark-able way to start driving customer-focused action at your company.

Many companies obsess too much about the number when they’re starting out.

However, the most successful companies never lose sight of the fact that the primary purpose of customer experience metrics is to drive customer-focused action.

Once you’re driving customer-focused action, you’ll start to actually reap the benefits of increased retention, expansion, and word of mouth.

One Oracle VP’s Three-Step Recipe for NPS Survey Success

Joshua Rossman is an NPS OG, having run NPS at eBay and McAfee, among other companies. He’s now Vice President, Customer Experience Strategy at Oracle.

Through his years of experience, Rossman has created a three-step system he uses to get the most out of customer experience surveys, including NPS. He’s been kind enough to give us permission to share it publicly.

Step 1: Ask an easy-to-answer anchor question first to improve response rates

This principle is standard for NPS, but powerful enough to use across other CX surveys.

Ask your broad question first, and get a quantitative rating. Making your first question easy to answer will improve your overall survey response rates.

Step 2: Get S-P-E-C-I-F-I-C with your open-ended ask

Rossman has found that the standard open-ended question, “Care to tell us why?” often leads to vague, inactionable responses (e.g. “It’s hard to use”).

He’s found that asking promoters for specific reasons they recommend — and non-promoters for specific ways to improve — leads to much more actionable feedback.

Here are the specific questions he recommends for brand-level NPS:

Promoters: “What is it that makes you most likely to recommend {{company}}?”

Non-promoters: “What is it we could do that would make you more likely to recommend us in the future?”

These questions ask more specific questions — and tend to get more specific answers.

Various platforms such as InMoment can help you automatically categorize your now-more-specific NPS verbatims!

Each company will want to tag their work in a way that makes the most sense to them, but Shaun Clowes, former Head of Growth at Atlassian, says that they used machine learning to tag their feedback into three categories: Reliability, Usability, and Functionality. They used the ratio of different complaints to understand, at a high level, where their product needed work.

Step 3: NPS’ Secret Third Question

Even with the more specific responses, you’ll hopefully get from the tweaks recommended in Step 2, not all B2B companies get such a high volume of responses that they can glean mathematically reliable responses from text alone.

One way to gain a deeper understanding of the factors that lead to an excellent (or poor) user experience is to follow questions about satisfaction with questions about various attributes of your brand. Ask a few extra questions with NPS and you can capture the overall sentiment for each area:

After you’ve captured these details you can then run a simple linear regression, which will tell you which factors most influence if a person is a promoter or a detractor.

Various versions of the linear regression technique were also mentioned by Allison Dickin, VP of User Research at UserLeap, and other experts.  Hearing them reinforce the power of this third question helps us get really excited about what we might do with it.

“Extra questions should be used judiciously,” counters Jessica Pfeifer, Chief Customer Officer at Wootric. “Think about it: When was the last time you responded thoughtfully to a multi-question survey?”

If you’re worried that such a long third step may lead to a negative user experience or lower response rates, a lighter option may be to ask the respondent to tell you what drove their score by selecting from a pick list of reasons.

If your follow-up question to detractors is, “What is the main thing we need to improve?” you could offer a picklist that includes product, support, training, and value.

Not only are you learning what’s driving your score overall, but you’re also generating groups of users to follow up based on their interests. For example, your customer support team can learn more by reaching out to detractors who cite “support” as an issue.

Drive Strategic Action with a Cross-Functional Cadence

You may have noticed that our first heading was about driving action on behalf of the customer.

We’re touching on it again because, ultimately, driving action on behalf of your customers should be the primary concern of an NPS program.

Driving tactical action on behalf of customers was something we were already doing well at PandaDoc, before talking to the experts. Getting NPS data into Slack and other systems has been a pillar of our NPS program — this helps us take immediate action on issues that surface in feedback. One example: reaching out to an unhappy detractor and quickly fixing the issue that her NPS feedback brought to our attention.

However, learning how many companies drive strategic action on behalf of the customer in the following way, was eye-opening:

• Collect customer feedback in a central repository (NPS, sales feedback, CS feedback, etc. — all combined together, somewhere like InMoment, UserVoice, or ProductBoard.
• Perform a 360° analysis of this data on a quarterly basis
• Set up a monthly cross-functional cadence to decide which action to drive, and to track progress and accountability on ongoing courses of action

Fictional Examples of Driving Strategic Action:

Product Team

Diagnosis: Self-serve onboarding is our most common NPS complaint.  People often come away without understanding our platform’s core concepts.

Initiative: Improve self-serve onboarding to teach core concepts of the platform.

Success Team 

Diagnosis: Feedback about CS indicates all roles except admins are quite happy. Admins specifically have trouble understanding how to set user permissions, and they’d rather avoid going through training to learn something so small.

Initiative: Create micro-videos that explain to admins on how to manage user permissions.

Support Team

Diagnosis: NPS feedback indicates enterprise customers are unhappy with the time it takes to resolve support interactions involving custom features. 

Initiative:  Route tickets from enterprise customers directly to senior agents who have the expertise and product knowledge to resolve their issues.

Marketing 

Diagnosis: Many of the leads we’re attracting cannot benefit from our core value proposition.

Initiative: Better align their SEM campaigns and landing pages with promises that the product can fulfill.

Your metrics should flow from your unique business strategy

NPS has been sold by some as the be-all / end-all metric of a customer-centricity program. But this approach can be harmful.

While NPS is often a great way to understand brand-level sentiment, it makes sense to layer on additional metrics as your CX program progresses.

Jessica Pfeifer at Wootric and Allison Dickin at UserLeap agree on the idea that your CX metrics should flow from what’s most critical to your business’ success.

“You’ll be able to benchmark and track trends over time when you complement NPS with established customer experience metrics like CSAT, PSAT, or Customer Effort Score at critical touchpoints in the customer journey,” says Pfeifer.

“For example, you might trigger a Customer Effort Score survey to gauge how easy it is for a user to achieve ‘first value.’ What is that critical milestone in your product? In PandaDoc’s case, it might be sending a document. Here at Wootric, it’s when a customer has live survey feedback flowing into their dashboard.”

Both took time to talk to us about questions that can be used in addition to (or as an alternative to) NPS. Here are some examples:

Example Non-NPS Questions

Takeaways

• NPS is a great way to get started with driving customer-centric action
• Use Josh Rossman’s three-part system to get the most out of your CX surveys, including NPS
• Use analysis and a cross-functional cadence to drive org-wide, customer-focused action
• As your business grows, layer on metrics that fit your specific business needs

This is just the tip of the iceberg for NPS, but we hope it will help your company squeeze the most out of your CX research program.

Hit me up on Twitter (@nathanlippi), and to let me know what’s worked well for you and your company!

So you’ve been reading up on Net Promoter Score. Your colleagues in the SaaS world tell you that it’s the best way to take your customers’ pulse. You’ve seen a few case studies claiming it’s the only number you need to measure.

It’s true that Net Promoter Score is a great way to engage with your customers and solicit tons of feedback. But it’s also true that there are quite a few nuances that result in a successful survey program.

As a SaaS company with SaaS customers like Zoom, DocuSign and Hubspot, we have a unique perspective on NPS in cloud software. To make the most of your time and energy, we’ve put together this list of things SaaS businesses should know before they dive into the NPS world. Read More…

Financial markets are sliding, a pandemic is spreading around the world, and every company is scrambling to respond to quickly changing circumstances. Planned investments that were intended to drive growth — like hiring, media spend and software purchases — are being reevaluated as business leaders are forced to triage what they need to do to weather the storm. We’re all in survival mode, but survival is about prioritizing what is most important.

And what is most important to a SaaS business at this moment?

It’s not toilet paper.

It’s our existing customers.

Now more than ever, customer experience is job #1. 

We think the SaaS businesses that focus on retaining customers and building loyalty are the ones that will survive and thrive in this uncertain climate. 

Of course, the question then becomes how do you retain customers and build loyalty?

Shift from a growth mindset to a retention mindset 

This may not hold true for every business we work with – Zoom, GrubHub, and the e-commerce toilet paper company Who Gives a Crap are having quite a moment. But most businesses are facing contraction because people don’t buy in a panic. Budgets are being trimmed everywhere, and customer success and renewal conversations must be deeply empathetic to this.

So, if a customer is achieving goals with your software, and you have other features and capabilities that will make them even more successful in 2020, then, by all means, paint a bold vision of an expanded partnership. But, if that isn’t the case, and they want to reduce or leave, don’t come across as tone-deaf. It’s likely that everyone in their company has been asked to find ways to trim spend.

That means that it’s even more important to know that your internal champion can confidently advocate for you – because you are delivering value. Step up your customer success initiatives. Make sure you and your clients are recording successes. And don’t be afraid to change the conversation from trying to get the customer to buy more, to showing him or her how the company can get more value from what they’ve already purchased.

Listen to your customers even more carefully – and respond

Even when you’re focusing on your existing customers, don’t make the mistake of thinking they’re the “same” customers – they’ve changed. We all have. Because our needs change in a downturn. Companies that are on the pulse of those changes by proactively listening are better poised to adapt, innovate, survive, and serve.  Make calls to key customers.  It’s even more important now than it usually is to listen and respond to concerns quickly. 

Take care of your people

Your employees, your teams, are the key to your customer relationships. They may be concerned about their health or the health of their parents, or grandparents. They may be struggling to find childcare options if their schools are shut down. They may be stressed about their 401K balance. Whatever it is, empathy and flexibility are going to be key – and so is prudent business planning. How can you plan to support your employees through these challenges?

Be a good citizen

For the collective good, and for the good of your brand, it’s so important right now to prioritize the good of the community and show conscientious, caring judgment. To do this, you may need to make some tough calls that hit your short-term profits, but protect people. We’re all making sacrifices – the New York Times dropped its paywall for coronavirus news and Zoom is giving K-12 schools free video conferencing. Is there a way your company can help people in need right now? You’ll be remembered for it. 

Establish a company policy of flexibility

Just as you have to be flexible with your employees and their quickly-changing challenges, you also have to be flexible as a company. For example, if a client calls customer support to request an extended payment plan, empower your support team to deviate from your standard policies and allow it. Be open to changing how you usually do things if it makes sense and shows compassion. You’ll likely prevent avoidable churn.

Jessica Pfeifer, Chief Customer Officer at Wootric, shared this recent story with our team:

“I just had a customer reach out about putting their subscription on hold. They operate in the hospitality sector which has been particularly hard hit. We offered to work out a plan to enable her to continue to get customer feedback during this critical time. Her response was ‘That would be amazing! Thank you!’ I know we’ve strengthened customer loyalty.”

Customers notice the companies that support them in difficult times, so be flexible when you can, and you’ll build loyalty for the future.

Close the loop with customers when they offer feedback

This may be built into your CX program already, but if not, now is the time to double-down on listening and responding to customers. Ensuring your customers feel heard and cared about in times of high stress carries more weight than when times are easy. So if a customer responds to one of your surveys, be sure to close the loop and let them know you value their time and will take appropriate action. 

Customer success managers can reach out one-on-one via email or phone, but that isn’t always practical. Closing the loop can be automated when you have your feedback readily available in systems like Intercom or Salesforce. Here’s a quick guide on how to automate closing the loop on customer feedback.

Improve customer experience at customer journey touchpoints

In SaaS, this often means using an NPS survey to gauge overall loyalty and surface any issues that may affect renewal. To get serious about retention, consider asking for feedback at critical SaaS journey points–after onboarding, support interactions, and during product/feature use.

This isn’t about quickly adding a slew of new surveys overnight; it’s about prioritizing improvements to moments that, if not successful, can sow the seeds of churn. Now is the time to double-down on understanding and improving the customer journey.

Also, remember to analyze the qualitative feedback from these surveys. A customer who is “satisfied”  but mentions a concern over price may now be at a higher risk of churn.

Focus product development on reducing friction for existing customers

In the software business, product experience is the all-important driver of customer experience. So, to foster customer loyalty, think about what you can do to create more ease for existing users.

Blake Barlett at OpenView says product-led growth is the key to success in the End User Era. In this era, end user annoyance spells opportunity. Think Slack vs. email, or Zoom vs. Hangouts.

In financially uncertain times, a product-led development philosophy can hold the key to faster end user adoption and increased retention. Tune into those day-to-day annoyances – they hold the key to retention.

Accelerate end user adoption

Happy end users make your application stickier, so if your champion is struggling to persuade others in their company to use your platform, you need to know why. You may need more in-app cues and guidance to make tasks easier. What is “so annoying” about your product? Ask your customers that, and you may find exactly what you need to reduce friction – which will pay off in retention.

Now is the time to deepen relationships and partnerships with promoters

Guneet Singh, Director of Customer Experience programs at Docusign, spoke about this in a recent Voice of the Customer webinar. He looks for champions among his promoters who have a common pain point, and then brings them together in councils that engage with DocuSign’s product team. Through this customer advocacy program, his customers learn from each other, get a first look at new product features, and provide valuable insights for the DocuSign product development roadmap.

How do you begin a customer advocacy program like this? Pay attention to customer requests and “start with small wins,” says Guneet. “If you complete the feature that a customer asks for, by listening and acting on their words, you’ve won that customer for life.” 

We couldn’t agree more!

The most valuable commitment we have is to our customers. And as much as we work to grow, to scale, to expand — it’s times like these where we have to remember to appreciate the people who already support us and show them support too. We’re all in this together.

What’s your biggest problem as a Product Dev professional? Too many demands and not enough time? Limited resources? Oddly enough, none of those topped the list for Hiten Shah’s crowd.

Hiten Shah (of KISSmetrics, Crazy Egg, and Quick Sprout fame) recently wrote in his newsletter that “the problems people have on Product teams fall into two main categories: Customer Feedback and Alignment.” This conclusion came after Hiten asked his readers to share their biggest product problems, and in more than 100 replies, those two themes emerged as the leaders.

Wootric helps customers gather, organize, categorize and analyze customer feedback – at volume – every day. And we’ve got a few insights into how Product teams can solve the issues that come with customer-centricity – while improving alignment at the same time.

Let’s go through the problems real Product professionals sent Hiten Shah point by point.

“Fast/Effective ways to quickly recap and synthesize qualitative research”

Qualitative data – ie. freeform responses versus ratings or multiple choice answers – are notoriously difficult to sift through and analyze. It’s only recently that, with advanced technology and machine learning, it’s become much easier to tag, sort, and assign sentiment to qualitative feedback at scale.

Tagging, in particular, is a huge time-saver when you switch from just manual tagging to auto-tagging. Tagging comments with their major themes is the first step towards conducting frequency analysis to identify trending topics – or find relevant feedback with a click.

Using an NPS survey with an open-ended comments section, for example, you might find that your ‘detractors’ (low scorers) comments tend to be tagged with “slow loading time” or you may see a specific feature request recurring.

Yep, modern customer feedback software should be able to deliver every comment with a feature request, for example, tagged and prioritized by frequency, from the highest-value customers, in about a second.

You can even use tags to route specifically tagged feedback straight to the appropriate department for follow-up. No need to hunt for bugs – the bugs will come to you! (Don’t they always?)

“It’s [customer feedback] very subjective and sometimes doesn’t have context, therefore I take it with a grain of salt, but engineers may not see it that way and want to address the feedback immediately.”

When your customer feedback comes primarily through surveys that *don’t* include open-ended responses (to gather all of that golden qualitative data), it’s impossible to get the context you need to evaluate the issue and possibly solve it.

But understanding the why behind NPS, CES and CSAT scores (to name a few) isn’t all the context you need to decide where to allot your time and resources.

You literally have to consider the source.

Is the feedback coming from a high-value, ideal client? Is your existing survey solution capable of identifying those markers?

Did you know that it’s even possible to target specific customer segments with survey campaigns?

And for even more context – you can target customer surveys based on product milestones. For example, you can set a CES survey to deploy after new feature use to find out how easy (or difficult) new customers think it is to use.

“Feedback overwhelm – how to prioritize what users want/need the most.”

An overwhelming number of customer comments can leave you feeling like you are trying to drink from a fire hydrant. It’s time to talk about the wonders of machine learning.

Historically, extracting insights from piles of unstructured feedback has been difficult, expensive and time-consuming. That is not the case today. When you need insight from feedback at scale, it is time to invest in text and sentiment analysis using software with natural language processing.

Machine learning has come a loooong way. Yes, algorithms must be trained to understand your company and customers, so chose a software vendor that will keep their team in the loop and ensure you’re getting good insights right off the bat. Then the software just gets better and better at telling you what is most important to your customers.

Wootric CXInsight™ combines natural language processing with sentiment analysis to categorize feedback based on what matters most for your customers. When you know why your customers love you — or don’t — prioritization becomes a much easier task.

“Having a regular cadence of customer interaction to develop insights and product intuition.”

Okay, there’s no excuse – this is so easily doable. You can set any CX survey you want to deploy on a regular basis, or, deploy after customers complete specific milestones. Having to go get customer feedback shouldn’t be something you have to think about. It should be automatic! Part of your daily, weekly, or monthly routine.

But, it’s only that easy if you’ve got software that makes it that easy – let’s be honest here. Modern customer feedback software can integrate with Slack, Intercom, or whatever you use, as well as deliver surveys to customers while they’re in your app, and deliver it to you tagged, sorted, and prioritized.

Regularly!

You can have your finger on the pulse of customer satisfaction and will know immediately if there’s any fluctuation. As an added bonus, give a pat on the back to whoever built an update or solution for customers so they can see the results in action!

“My main problem is to get to know our audience and talk directly to them.”

Surveys are great – we love them. But you know what? Even with a qualitative feedback field, a survey can’t take the place of a real, person-to-person conversation. And usually, the biggest barrier to having those conversations is making the time.

We can’t pick up the phone for you, but we can save you time. Enough time to schedule interviews with your customers and get even deeper insights that they may never tell you in writing.

“In Product we’re expected to be customer-centric. We’re supposed to get feedback and talk to customers all the time. It’s literally our day job. But that’s on top of making sure we’re focused on building the right things and helping our teams ship too.”

Here’s the thing, Product friends. You aren’t the only department that has to be “customer-centric” and talk to customers all the time and review steady streams of feedback. So to make this part of your job easier, you might have to reach out to other departments and make customer-centricity a multi-team effort.

If you have a Customer Success department, start there – you might find that the Customer Success Manager is your new BFF. They’re also talking to customers every day, and in many ways, they’re closer to the problems customers face than you are. Most CSMs would be delighted to build better relationships with their Product Dev departments, working together to answer the question “What can we do to help our customers achieve success?”

“It’s not easy and it isn’t getting easier. Customer feedback can come from anywhere: Customer support requests, live chats, social media, the sales team, customer reviews, competitor research, and more. Adding to the pile are the endless opinions about what to do with the feedback from people on our teams.”

It’s not easy – true. But it is getting easier to solve qualitative feedback issues with modern customer feedback software!

Sorry, we can’t help with the ‘too many cooks in the kitchen’ problem – that’s right up there with finding the cure for the common cold. We find that if you have to pick one source to guide product, NPS feedback is the going to be the most actionable.  That said, when it comes to gathering customer feedback from many sources into one, easily searchable place, modern technology comes to the rescue again.

What you want to look for is a customer feedback program that can pull all of customer comment sources together, like NPS or CSAT feedback, user interviews, support tickets, app store reviews, social and analyze those comments in a way that lets you see the big picture and slice & dice by theme, sentiment, survey date, and data source.

This guest post was written by Martin Ceisel, the lead Content Strategist at MindTouch. His hobbies include writing, writing, and writing some more. MindTouch is a self-service platform that helps companies improve support agent productivity, increase ticket deflection, and fuel self-service support.

A quick look at some Net Promoter Score benchmarks will quickly reveal a painful truth: bad NPS scores happen. It’s inevitable.

The worst response to your company’s detractors, though, is no response at all. So, how to best learn from bad Net Promoter Scores and use them to improve the customer experience?

Here are a few strategies to consider:

Do your research

Look at all of the support tickets your detractor customer has put in and read all the notes that your agents have written about these interactions. Review the goals they had when they initially became a customer. Check which help articles they may have read. This will give you important context when you close the loop with the customer.

Respond promptly and personally

Though the customers behind bad Net Promoter Scores might still be feeling the sting of their negative experience, receiving a prompt response to their NPS survey might help turn the tide. If nothing else, a personal response is an opportunity to take the NPS survey beyond a transactional call and response to an ongoing (and honest!) conversation. You’ll be surprised how much constructive feedback a simple “What can we do to improve your experience?” might unlock.

Segment response types

What customer group or business segment is driving the bulk of your bad Net Promoter Scores? One way to find out is to segment NPS scores to identify hotspots. You might find that a particular point in the customer journey, such as onboarding or renewal, is creating an inordinate number of detractors. Or maybe your NPS from product A is higher or lower than product B. Ask yourself why one group of customers is more successful than others. By categorizing responses, you can drill down and identify actionable takeaways. One way to see themes is to create reason codes, a method of categorizing responses so they can be organized and analyzed.

Don’t get tunnel vision

Remember that NPS is just one measure of customer sentiment. Don’t forget key metrics like customer effort (CES) and customer satisfaction score (CSAT). These, too, are important metrics that can lead you to the root cause of negative customer experiences. Regarding NPS specifically, consider trends in your industry. What are the NPS benchmarks you should be aiming for? This will help you decide how urgent an action to take—which bad Net Promoter Scores to prioritize first.

Because it’s about the whole customer experience

Tunnel vision makes for a good segue to my close: remember the reason we pay such close attention to customer sentiment. Perusing, parsing, and responding to bad Net Promoter Scores is about more than improving your company’s own internal metrics. It’s about improving the customer experience. If we can’t deliver low-effort customer experiences throughout the customer journey—if we don’t demonstrate a commitment to reading and responding to what our customers are telling us—we risk losing those customers entirely.

“What grade did you get?”

Do you remember getting asked that question in grade school? Or maybe you were the one asking it? Humans like to know how they’re doing compared to everyone else.

This carries over into customer experience as well. At Wootric, we advise companies on setting up an effective Net Promoter Score (NPS) program. We get asked questions about NPS industry benchmarks all the time.

In general, we believe focusing on an external NPS benchmark is not incredibly helpful.

The Net Promoter System is the quantification of customer loyalty and the process for improving it over time. The power of this system lies in the analysis of feedback and the action taken based on that analysis.

However, net promoter score benchmarks are still useful in certain cases, which is what this article is all about.

If you’re unfamiliar with NPS, here’s a quick rundown:

Net Promoter Score (NPS) is a customer loyalty metric between -100 and 100 that captures the propensity of a company’s customers to attract and refer new business or/and repeat business.

NPS also stands for the Net Promoter System®, which was built around the Net Promoter Score. It is a model that ties a corporation’s bottom line to customer happiness and loyalty.

In the NPS survey, customers rate their likelihood to recommend your company on a scale of 0-10. To get your Net Promoter Score, take the percentage of people who are happy and willing to recommend your product or service (those who respond with a 9 or 10) — “promoters”– and subtract the percentage of people who would not be willing to recommend your product or service — (score of 0-6) “detractors.”

For example, a +50 NPS means that the company has more than 50% promoters and less than 50% detractors, so generally an NPS score of +50 is, indeed, great! You may see scales out there that say +30 is a decent score, and that +80 or greater is the ultimate dream score.

To learn more about NPS, get the ebook, The Modern Guide to Winning Customers with Net Promoter Score, which teaches how to modernize your NPS program for growth and higher loyalty.

Net Promoter Score industry benchmarks

There are two different types of NPS: absolute and relative. Absolute NPS refers to the NPS in and of itself, and comparing the score with what is generally considered a “good” or “bad” score. Relative NPS is taking into account the average NPS within an industry, which takes into account the factors that could affect an average Net Promoter Score, and can change the NPS benchmarks you set.

While an absolute NPS goal is nice and simple, it can be helpful to take a look at what others in your industry have been able to achieve, since every industry is different and has unique relative NPS results. The relative Net Promoter Scores generally achieved in each industry help construct what are called the NPS industry benchmarks. NPS Industry benchmarks give you a way to evaluate your NPS relative to your competitors. They help control for factors that often create major differences in what is considered a good NPS score.

Oftentimes, other companies in your industry have established an average NPS for you to use as a net promoter score benchmark. If you make smartphones or other tech hardware, for example, companies like Apple have been tracking NPS for years.

To get averages and examples from your industry, try reports from the Fortune 500.

NPS Benchmark variance between industries

Let’s take a look at some examples of net promoter score benchmarks according to your industry.

Let’s say you have an NPS of +50. As we explained, that’s already pretty good! But if you’re a department store or specialty store, you are actually below the NPS benchmark (+62) for the industry.

Walmart pharmacies have an NPS score of +32. Considering the highest score is +100, you’d guess that they’d be lukewarm with this score, but I’m sure that the folks in charge of customer experience there are actually ecstatic. Walmart pharmacies have one of the highest NPS scores within the drug store & pharmacy industry.

Compare this number to the software industry, where +34 is the average. Becoming a leader in the software industry would mean having an NPS in the +60 range, like Salesforce (+66) and Adobe (+62).

Caveats for using NPS industry benchmarks

Unfortunately, NPS benchmark programs aren’t always as helpful as you’d hope. This comes down to the nature of surveying for feedback. There are so many contributing factors to an NPS benchmark, such as:

• Which channels you use to survey customers
• Demographics and habits of your customer base
• Customer tolerance levels
• The size of your competition
• The difficulty of building brand loyalty
• External circumstances (such as a global pandemic)
• When and how often you ask
• Whether you have enough data to be statistically significant or not

All of these factors can have varying effects on your overall NPS score. For example, your competitor may ask the NPS question within the context of a longer annual brand survey, while you survey using just the NPS question after a transaction. These will have different consequences for the feedback you gather. If you don’t have enough feedback coming in, your NPS may vary significantly from quarter to quarter or month to month.

Bear in mind, a ‘good NPS score’ doesn’t just depend on your industry, since it’s not difficult to game the system. It’s not always fair to compare your NPS score to another company’s NPS score because you don’t know their survey methods, or their employee compensation plans.

When competitive individuals are incentivized based on NPS score, things can get ugly.

A motivated person or company could improve their numbers by letting their customers know that positive feedback would mean a lot to them or by only showing the survey to customers who are positively inclined. They might offer incentives to customers to complete the survey. Clearly, the feedback received from these methods will lead to an inflated NPS score that is not a useful comparison for those using a more objective survey process.  

Setting an NPS goal if you don’t have a benchmark

If no Net Promoter Score benchmark exists for your industry, benchmark against yourself.

The great thing about NPS is that it is an actionable metric. It’s a number that you can rally the company around as a north star to guide improvement efforts.

“A good NPS score is one that is better than the last.”
– Jessica Pfeifer, CCO & Co-founder of Wootric

Remember, NPS isn’t just a score. It’s a system that’s meant to drive business improvement in product and customer experience. It helps you identify and close the loop with unhappy customers and solve their specific problems in real time.

Your goal is to boost customer loyalty and retention, and that happens by reading verbatim comments to understand the why behind the scores you receive. By making changes based on customer feedback, and responding quickly to detractors, you will naturally see your NPS improve. And gains in NPS correlate with revenue growth.

How to report NPS

After all this, you will want to report numbers to the rest of the team on a regular basis. NPS should be shared along with other monthly or quarterly metrics like revenue, new customers and customer churn.

We understand that, so here’s what we recommend:

• Instead of fixating on your score in the absolute sense, we recommend focusing on improving your score over time. Understand NPS as a trend over several periods, like if you were looking at a stock’s price.
• Determine the business goals of your NPS program, then report NPS in relation to the goals. For instance, if you are trying to improve retention, report NPS alongside churn data.
• Pay attention to trending topics in your verbatim responses. Reporting these topics will help everyone understand what’s important to your customers, and the pain points they experience. Share what customers love and what they don’t love about your company with internal stakeholders. Then you can work to make those points as frictionless as possible. 

Note: For startups, be sure to read and respond to every single comment. As you grow, you’ll start needing aggregate and to pull themes from customer comments. To automate that process, check out AI-powered text and sentiment analysis.

• Segment your Net Promoter Score by relevant customer groups. For example, this could be by user role (in the SaaS example above), geography, or size/frequency of purchase–whatever drives your business. This will help you pay close attention to groups that are critical to your business success. Learn more about segmentation here.
  
• If you want to compare your score to a competitor, choose a company in your industry that you admire and use their score as an aspirational benchmark. Many companies have volunteered their NPS scores to research and reports such as this one by the Fortune 500.

Measure NPS and work to improve it over time.  Dig into customer comments and close the loop with customers. You will learn their needs, and their pain points, and have plenty of guidance to make those improvements. Both your NPS and your customer retention rates are sure to improve. 

What is Empathy?

Empathy is the ability to understand and share the feelings of another. For Product Managers looking to improve customer experience (CX), that definition translates to doing more than understanding the user’s pain points, but also looking at the emotional landscape of what it’s like to use the product – when it is working, and when it isn’t working.

Empathetic Product Managers ask themselves:

• How does using the product make the customer feel?
• How does the customer want to feel when using your product? What would be the best possible emotional outcome for them?
• How do I ensure the product developers understand and take the customers’ needs into consideration in their process?

The answers to those questions affect every facet of business, from acquisition to retention. It’s how, through CX, you can generate rapid growth through word-of-mouth recommendations, and sustain your success with customers who never want to leave.

Tying Empathy into CX

Empathy is a soft skill, and while those are typically difficult to measure, the effects of empathetic product development can be seen in every CX metric: Customer satisfaction (CSAT), Customer Effort Score (CES) and Net Promoter Score (NPS).

Sinead Cochrane, Senior Product Researcher at Intercom, wrote “For product teams, empathy building activities such as observing research or doing customer support is often not considered ‘real work’. However, product teams that consistently keep customer needs in mind are able to maintain and evolve their products in ways that won’t negatively impact the user experience.”

For her, empathy for product departments means “When a customer tells you something is broken, you are able to imagine the impact it’s having on the job they’re trying to get done,” and, “you realize the emotional impact the problem is having on that person.”

But I think we can go further than just recognizing the emotional impact of problems. That’s scratching the surface of what having empathy for customers can mean for producing superior customer experience.

Because empathy shouldn’t be reduced to realizing customers feel bad when a product isn’t working for them. A whole new world opens up when you also consider how you can design your product, updates, and expansions to enhance positive emotions as well.

Here are the questions that lie at the heart of empathetic product management:

• How can you get more emotionally in sync with your customers?
• Which are the most important negative emotional outcomes to manage?
• Which emotions should you seek to heighten (and how?)

To answer these questions, try these empathy building exercises.

Empathy Building Exercises for Product Managers and their Teams

1. Listen actively to discover underlying needs and emotional motivations

Often relegated to customer service and customer success departments, ‘active listening’ to find out why your customers use your product and what they really want to achieve is very important. You can’t get the depth and honesty of answers by just sending out a survey – this works much better if you do phone, Zoom or in-person interviews. In fact, Roman Pichler recommends product managers meet real users on a regular basis. You may find that your assumptions of why customers use your product aren’t accurate, or don’t tell nearly enough of the story.

“At first, our assumption was that they wanted to make more money. That often was true, but frequently we heard something different. Many simply wanted to maintain the business but run it more efficiently so they could have more free time (we heard about golfing on Fridays more than once). Others wanted to build a sustainable business they could pass on to their son or daughter.” – Jim Semick, Founder & Chief Strategist at ProductPlan

To get down to customers’ real motivations, ask open-ended questions beginning with “why” and “how.” Then make sure to record their answers in their own words (you can hand those assets to your copywriters for later use).

2. Use your own product

Empathy is often described as ‘putting yourself in someone else’s shoes’ – and there’s no better way to do this for a product manager than to actually use the product, just like any user would. You’ll empathize with users’ frustrations as you experience your own product’s shortcomings and hopefully find moments where it’s possible to create more delight.

But always keep in mind – you are not the average user. You’ll still need to listen to your users to get a complete picture of how they feel, and what problems they perceive as being severely aggravating.

3. Share verbatim comments

Someone, somewhere, is tracking customer experience metrics, sending out surveys, and collecting the answers. That someone might even be you. When reading users’ written responses, don’t just look for problems to solve and ignore the positive comments. Read them for emotion and see what conclusions you can draw about what people are feeling, and want to feel.

Pick a few relevant verbatim comments to bring to the rest of the product team. Reading these comments often helps engineers and designers feel the same joy or frustration as their users. This new emotional understanding will help you evangelize CX as a priority with everyone.

4. Mine your qualitative data and quantify customer sentiment

Those open-ended response answers are a goldmine for user research that can alert you to problems – and give you hints into the customer’s emotional state of mind. However, once you are getting more than a hundred comments a month, seeing the forest for the trees can be a difficult exercise. Qualitative feedback is notoriously tough to quantify, but it is now possible and easy to quantify sentiment with the help of machine learning.

AI-powered platforms, like InMoment CXInsight™, automatically sort your customer comments into themes while simultaneously assigning positive or negative sentiment. This provides you with a big picture understanding of how customers feel about your product and why. Categories of feedback vary by business sector and business model–payment processes & delivery for e-commerce, perhaps, while UX and usability may surface for SaaS products. Quantifying the sentiment of what your customers are talking about can help you track emotional trends over time. Presenting this kind of data alongside verbatim comments connects customer emotion with real business consequence.  

5. Set empathy KPIs

What gets measured gets done, and adding empathy into your product development work is no different. The KPIs for empathy may look a little different than your typical performance indicators, but the good news is: They’re not difficult to get. You’ll find key performance indicators like NPS, CES and CSAT are a good start, and comments in the open-ended questions can give you insight into the metric. Start identifying what kinds of ratings and qualitative answers correlate to genuinely happy customers – and frustrated customers likely to churn.

6. Chart out an empathy map

You’ve done your user journey, but even though it’s part of the buyer persona building process, you may not have done an empathy map.

• What your user sees – on competitors’ websites, common visuals in their industries, maybe what they enjoy watching or reading
• What your user says – how they measure success, what they say they want, what they say about your product
• What your user hears – what their influencers are saying, not just about your product, but about their jobs and what constitutes success, what they enjoy, what they don’t like about their experiences with your competitors, etc.
• What your users think and feel – worries, aspirations, what they really want, what really annoys them

Notice how the empathy map includes business/industry-specific observations, but also branches out into the user’s personal life and larger environment. People are not their jobs – or even their ‘jobs to be done.’ For true empathy, you have to look at the whole person.

This is a great activity to get other teams involved in – consider hosting a meeting with Customer Success, Sales, Marketing and Customer Service for a wider scope of insights.

Activities involving multiple teams help to build a shared understanding of your customers’ experiences that can strengthen the whole company.

7. Add happy moments to your Customer Journey Map

You’ve probably mapped out your customer/user journey, but you probably didn’t include this: Happy moments. See if you can take your old customer journey map and mark the points where positive, fun, delightful things happen. Can’t think of any? Then you have some serious CX work to do!

And of course, also note points where you’ve observed friction, difficulties, and problems, and address those in the order of biggest impact + easiest to implement.

8. Work in Customer Support for an afternoon

Whether that means answering the live chat questions, picking up the phone, or monitoring your product’s customer Slack channel, try out being the Customer Support agent for an afternoon to and put yourself on the front lines! There’s no better way to find problems than to let customers tell you exactly – and in great detail – what they are. And they’ll likely throw in how frustrated it makes them feel too.

9. Build a prototype to test your emotional hypotheses

By now, you probably have a few ideas on how you can improve the customer experience, and it might be time to test those theories. Create a prototype for a select group of qualified users to try (and react to). And, if possible, have them test the prototype in a testing facility that allows you to observe their reactions as they use your product.

If there is a Golden Rule for empathy, it’s a simple one: Forget your assumptions and be genuinely interested and curious about what people are feeling (not just what they’re doing) while using your product. Empathy is a learned skill that needs practice so don’t forget to try out these empathy exercises on a frequent basis for enhanced customer experience.  

To quote Maya Angelou: “People will forget what you said, people will forget what you did, but people will never forget how you made them feel.”

Data is the beating pulse of business, but customer data is more like DNA. Customer data, if we’re using it right, directs how we grow and what we develop. But what happens if that customer data becomes corrupted by our own bias?

We can’t grow or develop in the ways we need to.

But what is bias exactly? Where does it come from?

The most prevalent bias is, perhaps, confirmation bias – seeking out data that confirms our existing beliefs.

In an early study of confirmation bias, young children were asked what features in a sports ball are important to the quality of a player’s serve. Some said size, others said material, some dismissed color as a factor – but once they’d made up their minds, they failed to acknowledge evidence that was contrary to their theory – or explained away evidence that didn’t fit.

But what’s worse, especially for those of us using data to steer our businesses, is that confirmation bias caused them to not generate alternate theories unless someone asked them to. They missed exploring and finding other possibilities.

There are other types of bias too, including:

Algorithmic bias – When the data used to teach an AI machine learning system reflects the implicit values of the humans involved in collecting, selecting and using that data. You might remember the 2015 uproar around Google’s image recognition AI algorithm that auto-tagged photos of black people as gorillas? Yes, that happened. And in 2009, Nikon’s image recognition algorithms consistently asked Asian users if they were blinking.

Survivorship bias – When the data analyzed only comes from success stories.

Sample bias – When the population you collect data from doesn’t accurately reflect the population you’re trying to learn about.

Avoiding bias when gathering, analyzing and acting on data is impossible. Bias creeps in with assumptions, instincts, guesses, and ‘logical’ conclusions – and mostly, we don’t even know they exist until someone without those particular biases point them out.

But, while we can’t escape biases, we can try our best to account for them when we collect, analyze and interpret data.

How to fight bias in your data

In Forrester’s The illusion of insights recording, Forrester Vice President and Research Director Sri Sridharan makes three recommendations to reduce bias in data.

She says to “triangulate insight” by using multiple methods of arriving at an insight, and cross-validation. For example, pairing behavioral data with feedback from customer surveys to see if you arrive at the same or similar answers.

Her second piece of advice is to create a “self-correcting system of insights” that connects customer data with an effective action to create a closed loop of action, learning, and optimization. Essentially, this means testing the data by taking action and iterating based on how well you succeed in addressing the issue.

Tracking a ‘North Star’ metric like NPS or CSAT over time can be very helpful in confirming whether the changes you make are having the desired effect.

Sri’s third piece of advice is to “show your work to build trust” both internally and with customers. Your customers will be quick to correct you if your insights don’t hold true for them – and you have the bonus of showing them how hard you’re working to make sure they have what they need to succeed.

But there is also the potential for bias to happen before any of these fixes can be made – especially in Customer Discovery.

Bias in Customer Discovery, Before You’ve Even Gotten to the Data

Bias in whom you ask

Who you survey, interview or meet with can bias your results. This is called “sample bias” – but it can also turn into confirmation bias. Sample bias happens when some members of the intended population are less likely to be included than others. Think of all of the different segments of users you have – what would happen if you only surveyed one of those segments? You would get responses that don’t work equally well for all of your customers.

This can slide into confirmation bias if the population you select is more likely to give you the answers you want to hear.

And, there’s also the risk of “survivorship bias,” if the people you’re surveying are the customers who are still with you, rather than the users who have churned. Current users are much easier to collect data from, and while they can give you important insights, they can’t tell you why your churned customers left.

Bias in how you ask

How you frame questions can have a dramatic effect on the responses. In fact, by the wording you use in a survey, or even your tone of voice in a phone interview or facial expressions in an in-person interview, you can effectively steer the conversation to deliver exactly the answers you’re hoping to hear. Many of the words we use have positive or negative associations that cause people to react accordingly.

Biased question: How much do you like the color blue? (This presupposes they like the color blue at all)

Unbiased question: How does the color blue make you feel? (A much more neutral phrasing)

Or, if you aren’t specific enough about the information you want, you risk confusing your respondent and getting answers that aren’t at all helpful. Unless you have a professional market researcher on staff, you may want to stick with established questions like NPS and CSAT.

Bias in what you ask first – and last

The order of the questions you ask can also bias your results, and you’ll need to review your question order carefully to make sure the sequence doesn’t cause biased responses. Typically, you should ask general questions before specific ones, ask positive questions before negative ones, and ask questions about behavior before questions about attitude.

Bias in when you ask

Holidays and the summer months, when families often take their vacations, can be problematic for both response rates and sample bias. For example, if you send a survey during religious holidays, you’ll likely get different responses rates from different groups of people, who may or may not be taking that time off.  Be aware of your timing, including if you’re sending surveys during deadline rushes, before or after holidays, or other significant patterns that may affect who responds and how they respond. To be on the safe side, don’t send your survey at the same time every year – send a few, at different times, to get the most accurate feedback.

Objective Data Leads to More Accurate, More Valuable Insight

Sherlock Holmes famously tells Dr. Watson that he never forms a theory before he gathers all of the facts. But he’s much better at disassociating himself from the results than most of us are (and he’s fictional). Bias has a way of seeping into our results, and how we view and react to our results. But, when we put bias-countering measures in place, like gathering data from different sources, using different types of data, and checking our work through a process of action and iteration, we can get to the truth in the end.

Customer experience professionals live in a world overflowing with data. Sitting on that wealth of information is frustrating when you know it has incredible potential.

If you are tracking CX metrics, like NPS or CSAT, the numbers help you quantify customer loyalty and satisfaction. But it’s the customer comments that come with those surveys, all of that rich qualitative data, that give you invaluable context for why customers feel the way they do.

Until now, it’s been difficult to analyze qualitative data because it is so unstructured.

This is where tagging comes in.

Using software to analyze qualitative data

Modern customer feedback software comes with the ability to tag customer comments. Tagging feedback has two functional goals: Routing and Insight.

Routing:

Creating a tag for specific stakeholders, e.g. “product”, quickly sorts feedback to be routed to the correct teams for follow-up. Product teams can simply click a button to see verbatim comments regarding feature requests and support teams can be more proactive by checking for comments under a “bug” tag.

Insight:

Tagging comments by relation to product, website, or customer experience helps themes emerge. For example, you may see that most of your detractors are tagged with “shipping” or “price”. This will help you prioritize and address issues in real-time.

Tagging comments manually doesn’t scale, however.

If you are receiving less than 100 comments a month, manually tagging comments can work. But customer comments can pile up just like emails in your inbox. Constant monitoring results in little else getting done. When you find yourself drowning in responses, CX feedback can feel overwhelming — just like your inbox.

This is where using software to auto-tag customer comments saves the day.

Auto-tagging gives you real-time categorization of large quantities text feedback

Auto-tagging automatically sorts qualitative comments for you using AI-powered text analysis, and it happens in real-time. This helps you surface themes and see trends that the human brain has trouble processing on its own.

For example, you may find that pricing issues are mentioned in 80% of your detractor comments in the past couple months, or a new feature is mentioned in 65% of your promoter comments since it launched.

Auto-tagging serves as a dynamic tool to quickly sort massive amounts of feedback for routing to the appropriate teams for insight and immediate follow-up.

We’ve provided the first steps and some suggestions to start auto-tagging in real-time.

How to set up text-match Auto-tags

The time you save by setting up an auto-tagging system can be spent taking action based on the insight lifted out of your survey feedback.

If you aren’t using machine learning software, here are the steps to take in planning your text-match auto-tagging system and some suggestions to get you started.

First, Some Questions to Ask Yourself

When you start to tag your feedback, read every comment you receive in a period of time, perhaps a week or a month, and consider the following:

• What topics/features/issues stand out in your comments?

For example, you may see that many of your customers talk about your Support team’s response time, or the value your product/service has brought to them. These general themes will serve as jumping off points for brainstorming tags and keywords.

• Is there industry or business specific vocabulary or jargon that you might want to track?

For SaaS companies, you may want to include terms like “dashboard”, “widget”, or “in-app” as tags or as text-match keywords. Oftentimes, these terms will be abbreviated, like UI for “user interface”. 

You can even choose to create tags for team members to alert them whenever they are mentioned by name. This might be helpful for a customer support agent who wants to see what customers are saying about their interactions.

As you read through your sample of comments, make a note of the words and phrases you spot customers using. They may be using different terms than the language you and your colleagues use as professionals in your industry.

• Which teams will you be sending customer feedback to and what terms are relevant to them?

You want to be routing comments to the right teams. For example, a product development team will be interested in comments about user interface, integrations, or feature requests while your support or success team may be more concerned with bugs or implementation.

Nested Tags or Parent-Child Tags for Tag Hierarchy (SaaS example)

Once you’ve answered these questions, start grouping specific terms under broader terms. This is going to help you create hierarchy within your tags, also called nested tags.

Nested tags are labels associated by a hierarchy. The ‘sub-tag’ or ‘child tag’ is a tag that is more specific and can be categorized under a ‘parent tag’.

When any of the ‘child-tags’ are text-matched to a comment, feedback platforms will also tag that comment with the corresponding ‘parent tag’. Comments tagged with only the ‘parent tag’ do not include any of the words associated with any of the ‘child-tags’.

This allows you to pull comments that mention any of the specific integrations through the child-tags. At the same time, the broader “integrations” tag pulls comments that mention integrations in general, e.g. suggested integrations from our customers.

Choosing Text-Match Keywords or Keyphrases

For auto-tagging, it is important to choose the right words or phrases to match the tag to the comment. Text-match tags use an “exact match” rule for automation.

This is where having read through some of your current open-ended feedback is useful. You’ve seen the specific words that your customers tend to use when writing about different issues. It may also be helpful to use a thesaurus to come up with synonyms for the words or phrases you choose to match on.

Remember that text-match is very literal, so you will need to include variations on the words and phrases you choose. For example, an “implementation” tag should match on “implement”, “implemented”, “implementation”, and “setup”, as well as “set-up”.

Suggestions

We’ve compiled a list of auto-tags that are commonly used by SaaS businesses. You may be able to use some of these in other industries as well.

As you start to receive feedback you should refine your tags to be more specific to your business needs.

Here’s a list of common tags for SaaS companies to start with:

Human Review: Manually Tagging for Refinement

Monitor your feedback for a couple weeks after you set up your auto-tagging system. If a comment should be tagged, but isn’t, add more keywords to the text-match tag. Manually tag any comments that are difficult to text-match.

A good example would be a comment like “I tried to connect your software to my CRM but it didn’t work.” This comment is clearly related to integration, but text-matching wouldn’t catch this. After manually tagging this comment, you can then add “connect your software” as a keyphrase to the integration tag.

Human review becomes a tool for refining your existing auto-tags, instead of the main workhorse. As time passes, you’ll spend your time scanning for edge cases and new issues or topics that require a new auto-tag.

Do this check periodically to ensure your insight is accurate. Maintaining your valuable tagging system will save you time in the future.

Feedback Routing & Driving Action

Surveying customers is the first phase in your transformation into a more customer-centric company, but you will plateau if you sit on the feedback. Setting up an auto-tagging system means feedback is sent to relevant teams in your organization in real-time. Trends are lifted more easily from qualitative feedback, and your customer-centric organization will be empowered to actively pursue customer happiness.

When was the last time you completed the long survey you ask your customers to fill out? This is a painfully obvious (and obviously painful) exercise you can do to assess the customer experience of your surveys.  If the survey is long, you will probably find it a boring, tedious task to parse and answer the questions. Impatience grows as you face a seemingly endless list of attributes to assess. 

If this is what you are subjecting your customers to, know that you aren’t alone. Many companies are content with the status quo of traditional, bi-annual, 10+ question surveys, or they simply aren’t aware of alternatives.

But times have changed — and your customers aren’t having it.

Traditional, long surveys are a lose-lose situation

Not only do multi-question surveys have the potential to irritate customers, they have disadvantages for business as well.

 You are not hearing from enough customers.  Completion rates are abysmal. Studies show that the longer a survey is, the higher the chance of decreased, delayed, hasty or slapdash responses. So, the information you are getting from customers who are willing to run this gauntlet may not be thoughtful.  

Not hearing from customers often enough. Surveying once or twice a year means you can only react to feedback once or twice a year! In a quickly changing market, this is unacceptable. More agile competitors are going to leave you in the dust.

What can you do to solve this lose-lose situation? Modernize your feedback methodology with microsurveys.

What is a microsurvey?

Microsurveys take a well established, standardized question and use it as the first in a two-step survey. This first question can be used to measure Net Promoter Score, Customer Satisfaction Score, and Customer Effort Score, providing you with quantitative feedback. The second step then provides a way for respondents to give open-ended feedback explaining their score.

Here is an example using an NPS microsurvey shown to a customer who is logged into a SaaS application. A similar microsurvey can also be delivered via email, mobile, or SMS.

Your first reaction might be “How can I possibly get all the information I need with such a short, open-ended survey?  And, how can I make sense of all of the qualitative responses?

Let us walk you through how you can get what you need — and more.

Advantages of always-on microsurveys 

Microsurvey design looks at feedback collection from the customer’s point of view — it should be easy, fast, and relevant. The results are a significantly improved customer experience. Microsurveys provide three key benefits to you:

• Real-time trends
• High response rates
• Better insights

Real time so you never miss a trend:

With support of a customer experience software platform, it becomes easy to survey customers throughout the customer journey.  You can forgo your annual survey campaign and get a on-going pulse of real-time feedback on journey points.  Shortening your surveys allows you to ask customers for feedback more often. By asking the right question at the right time, you increase the chance that an individual will respond to your surveys. Deploying microsurveys across the entire customer journey will bring you both a bird’s eye view of the health of your account and detailed, actionable insights at each touchpoint.

High response rates means you hear from more customers:

Response rates can be as high as 60% for microsurveys, and typically exceed 25%. These numbers can seem miraculous compared to the significantly lower rates that long-form surveys attain. By asking a single question in the right channel at the right time, customer are more willing to give feedback.

Better insights:

Microsurvey responses will reflect what is important and relevant to your customers. Because you are no longer leading the respondent, you will learn things you wouldn’t otherwise learn. The qualitative feedback you receive is rich with context and potential to drive your business priorities.

Now, all of this may sound good but there are still barriers to making the switch, right?

Reasons why you are still using long form surveys

I can’t aggregate survey results when feedback is open-ended!

The advantage of endless Likert scale questions is that responses on a wide range of topics and attributes can be tallied and metricized.   This makes things easier for you on the back end. However, every time a customer must chose a response from a range of values, you are putting the onus of quantification on him or her. You risk asking them to evaluate something they do not know or care about.  Response quality, completion rates, and customer experience all suffer.

A modern approach is to save your scale questions for established CX metric questions like Net Promoter Score, “How likely are you to recommend [business] to friends and colleagues?”, and take the support of machine learning technology to quantify opened survey responses.   

Today, you can take the burden of quantification off of customers and place it squarely on machine learning software. In the past, getting insights from large quantities of qualitative data has been hard, if not impossible. Technology is now available to auto-categorize all of that rich, qualitative feedback. Auto-tagging and sentiment analysis have come a long way!

For example, this dashboard screenshot shows an analysis of auto-categorized NPS feedback. Auto-tagging reveals themes in qualitative comments so you can know what promoters, passives and detractors are talking about in real time.  

I need to ask a series of questions to get important information from our customers.

Every question you add is less likely to be answered with your respondent’s full attention and engagement. Asking a single scale question and an open ended question captures high quality data that is both qualitative and quantitative.

It feels counterintuitive to open up feedback to be a free-for-all; however, customers want to tell you what’s on their mind at the time you survey them. Asking exclusively about what is important to you is frustrating for the customer. Like the saying goes, you can lead a horse to water, but you can’t make it drink.

Getting the information you want is less obtrusive if you send customers a short survey at the right time. For example, you can send an microsurvey asking about how easy a transaction was to complete or how easy a feature was to use. Customers no longer have to reach into the depths of their memory to retrieve their impressions because they just completed the task you are asking about.

Asking for feedback at touch points over time, in the right context, creates a story of your customers’ journey and allows you to see trends, just like how thousands of photos can be combined to create beautiful stop-motion animation.

Beware of using incentives to make up for poor response rates, you will find a higher percentage of “satisficers”, or respondents who select answer options quickly and thoughtlessly to get to the incentive you promised them for “completing” their survey.

Of course, there is a time and place for long surveys.

There is nothing wrong with using a lengthy survey when you really need to — and there will be times when an in-depth questionnaire is appropriate. Here are two examples:

Annual “Brand” survey. Our customers use microsurveys to keep a finger on the pulse of their entire customer base throughout the year for customer journey feedback. Some also use an annual brand survey that supplements by asking many in-depth questions. Even though response rates for this survey may be low, they know they will hear from their most engaged customers on a variety of topics. And, with their microsurvey program,  they still get feedback from everyone else.

User interviews. Product teams may conduct focus groups or interviews to get more sophisticated feedback on feature use, build out an understanding of use cases, and create detailed personas. Microsurveys such as NPS help narrow down who should be included in these focus groups and who would be open to being interviewed.

How to start? Shift your Net Promoter Score program to microsurveys.

If you want to try real-time microsurveys as a baby step towards modernizing your feedback program, use always-on NPS microsurveys as one component of your feedback strategy. You’ll still send out your long, in-depth survey to decision makers like you always have, but now with an early warning system to help you proactively keep your most important accounts.

Entelo was able to double their survey response rate with this method, using NPS microsurveys for a better understanding of customer health. The real-time feedback also meant fewer surprises and easier prioritization when it came to addressing customers’ problems.