How to Train AI to Analyze Your Customer Feedback

Customer comments are the lifeblood of any CX program, giving you the “why” behind customers’ NPS, CES, and CSAT scores. But until recently, it’s been nearly impossible to make sense of feedback from hundreds of customers at a time. Using artificial intelligence (AI) to automate text analysis gives you the consistent and fast insights you need, at scale.

That said, automated text analysis isn’t just about technology. Humans need to put in the time upfront to teach the machine, by providing an accurately tagged set of feedback for AI to work from. The quality of that training data sets up the quality of your text analytics results, or as the old saying goes “garbage in, garbage out”.

Let’s look at what you need to be successful with automated text analytics. We’ll dig into the basics of text analytics, the inconsistencies of manual tagging, and how to create good training data and models.

A quick primer on AI training sets

Analyzing customer feedback from unstructured text can be complex. In one sentence a customer may talk about a variety of topics, offering negative, neutral, or positive feedback (sentiment) about each of them. It’s the job of machine learning to recognize what the customer is talking about and identify how they are feeling about those topics. With text analytics, it only takes an instant to:

Tag comments / categorize themes
Assign sentiment to each of the tags and the comment overall
Aggregate results to find insights

Text analyzed for sentiment and themes

Again, machine learning is only as good and accurate as the data set you (the humans) provide to train the algorithm. So you need to do it right.

At Wootric, we have a lot of experience helping teams create training datasets. While we have sets of tags that are specific to various industry verticals, we also build custom machine learning models for many customers. Custom models are helpful for companies that are in a new vertical or a unique business.

Training model process

For the most part, companies have a good feel for what their users/customers are talking about and the topic tags they need. If they’re not sure, we can analyze their data and work with them to help them think through a set of tags to get them started. Once they have a set of agreed tags, they start creating the training data.

The process for creating this training dataset goes something like this.

Decide what tags are important to your business
Create definitions for those tags so everyone knows exactly what the tags mean
Pull 100-200 customer comments

Our customer assembles a team of at least 3-5 people who independently review each comment and determine:

If the comment sentiment is overall positive, neutral, or negative
Which tags apply to that comment

That last point is where things get interesting for a data analyst like me.

Manual tagging: an inconsistent truth

Many companies still believe having human tagging and analysis is superior to AI. We’ve seen one employee hired full-time to pour over spreadsheets, organizing data and pulling insights, which takes A LOT of time. Other companies bring in a team of people (the interns!), which introduces inconsistencies. Not only is it expensive and time-intensive, manual tagging isn’t necessarily accurate.

These same inconsistencies appear when creating training datasets because the process starts with manual tagging. Customer teams creating training data are always surprised by the level of disagreements on “defined” tags. It can take a few rounds of work to iron these out.

Tag definitions vary

Not all tags carry the same level of complexity. Some tags make it easy for people to agree upon a definition, while others may be more ill-defined. Vague tags tend to invite more disagreement between human labelers who label the same dataset independently.

Let’s look at a couple of examples from the software industry:

“MOBILE” — applied to any feedback containing references to a mobile app or website functionality. This should be straightforward for a group of human labelers to apply similarly, and would most likely only result in a few disagreements between them.
“USER EXPERIENCE” — a more complex phrase with many different definitions of what could be included in a user’s experience. When a comment mentions search functionality, is that UX? How about when they say something like “While using the search bar, I found information on…“? Or even “Great product”? Because there is so little clarity on what fits in this category, the training team will surely disagree, leading to more rounds of tagging and defining.

The good news is that at the end of the process, after a few rounds of defining the tags and applying them, the team REALLY knows what is meant by a given tag. The definition is sharper and less open to interpretation. This makes the machine learning categorization more meaningful, and more actionable for your company, which leads to an improved customer experience.

Getting to a good training model

Let’s look at a real-world example of creating a training set, and the level of label agreement between the people creating those labels.

A recent enterprise customer used 8 human labelers on the same initial set of 100 comments, and we then evaluated the labeler-agreement of each tag. During this exercise, each labeler worked independently , and we charted the agreement scores.

In the following two charts, the number in each cell represents the strength of the agreement from 0 – 1 between two labelers (F1-Score calculated from Precision and Recall values).

1.0 would be the ideal labeler agreement value.
0 means no agreement.

The lower row of the chart contains the average agreement across all cells for that labeler.

In Figure 1, it’s clear that the tag is fairly well-defined, which results in an overall average labeler agreement of ~0.83. This is on the higher end of what we typically see.

Figure 1 – agreement on a well-defined tag

In other words, even a well-defined tag doesn’t garner complete agreement between labelers. Labeler 5, our most effective labeler, only scored a 0.85 average. Labeler 1 and 8, with an F-1 score of .75, didn’t apply the tag, in the same way, a significant portion of the time. But it’s still considered successful.

Now, look at Figure 2, which shows the first-round results of the team’s effort to consistently apply a more complex tag. It resulted in an overall average labeler agreement of only ~0.43.

Figure 2 – disagreement on a vague tag

For the same group of labelers tagging at the same time, two different tags demonstrate nearly 2x difference in overall agreement — showing again that even we humans aren’t as good at manually categorizing comments as we would like to think.

Even when teams agree on 1) what tags to use and 2) the definition of each tag, they don’t necessarily wind up applying tags in the same way. It takes a few rounds for teams to come close enough to consensus to be useful for machine learning.

Ready for autocategorization

Text analytics is not a perfect science. When are the label agreement results ready for prime time? Typically, we consider a model good enough to deploy once the F-1 score is around 0.6 (give or take a bit based on other factors). Like most things in life, when you invest more time upfront — in this case boosting F-1s with additional rounds of tagging/defining — you typically end up with better results.

Makes sense of customer feedback with InMoment CXInsight text analytics.

Back to Blog

CX Tipping Point: 7 Signs You Need Text & Sentiment Analytics

The potential for machine learning to elevate the customer experience has everyone buzzing. AI-powered text and sentiment analysis can be an incredible solution for specific problems that CX pros face.

But how do you know when the time is right to move to the next level of CX? Are there new tools you can purchase to step your game up? How do you know they’ll be worth it?

There are clear signs that your CX program is ready for, and your company could quickly benefit from, text and sentiment analysis. And we’ll delve into them here.

Before we get going, some definitions:

Text analysis takes qualitative customer comments and determines relevant themes. Software companies might see themes such as ‘feature request’, ‘bug’, or ‘pricing’. This allows you to quickly see what your customers are focusing on, and then dive in to see what they’re specifically saying about each topic.
Sentiment analysis offers micro and macro insights into how your customers are feeling about your company and products. It determines whether the text received for each text theme is positive, negative, or neutral. It also analyzes the comment as a whole, assigning sentiment to the entire verbatim text.

Let’s look at the 7 signs text and sentiment analytics will be worth the investment for your company.

1. You have a mature or quickly-maturing CX program.

Those of you considering text and sentiment analytics probably already have a few key elements in place:

A customer experience strategy and a Voice of Customer listening system
A C-suite sponsor who has been fostering a customer-centric culture across the whole company with NPS as the guiding star
A system asking for feedback through the entire customer journey

Now that you have a relatively mature CX program, you’re wondering how to extract even more value out of it.

2. You receive 500+ comments per month (or you’re headed there.)

Ideally, you want to listen to all of your customers – not just a sample or the first to respond. In reality, at a certain point the sheer volume of incoming customer feedback is more than a CX program can handle without an upgrade. You know this is the case when:

You feel excitement and dread regarding the amount of feedback you receive.
You’re anticipating a whole lot more comments soon.
You’ve even had to cap the number of comments you receive in a day to avoid being overwhelmed with the task of organizing and responding to everyone.

Overwhelming amounts of feedback is an amazing problem to have, but a problem nonetheless. Using text and sentiment analytics, you can turn unstructured qualitative feedback, like NPS comments, into organized insight in a matter of minutes.

Text and sentiment analytics allow you to analyze customer feedback using Natural Language Processing, looking something like this:

Read Google’s case study on Wootric and Natural Language Processing here.

By combining text and sentiment analytics, you can search negative comments and quickly assess, for example, that 80% of your negative comments are about pricing. Or 45% of your customers in the Northeast region are talking about slow delivery times. That summary lets you know where to focus resources, and how quickly you need to make the change relative to other company priorities.

3. You’re sitting on a goldmine of feedback, but unable to get actionable insights.

Do you have a backlog of comments waiting to be read and sorted? Or maybe you’ve skimmed a few comments to answer the urgent ones, but you keep putting off the others.

One of our clients came to us with NPS survey comments from thousands of users. But rather than mining that information, they were running focus groups to prioritize feature requests because it was easier. They were duplicating efforts to get information they already had but couldn’t access and act on.

“The two biggest mistakes [in CX] are not doing qualitative research in the first place and then not putting it to use.” –Morgan Brown, Product Manager at Facebook and coauthor of ‘Hacking Growth’

If you’re feeling this pain, it’s time to automatically mine the insight from that pile of comments you’ve been sitting on. Turn anecdotes and hunches that you’ve got about your customer experience into evidence-backed insight by using. And do it quickly with text and sentiment analytics.

Source: CXInsight™ Dashboard

Sliced and diced organized feedback is easily available with many platforms that offer text and sentiment analytics. Doing this can help you understand the root cause of trends – like the needs of different customer personas or geographic regions – more comprehensively.

4. Manual feedback organization & categorization is insightful, but painfully slow.

While some customers duplicate efforts between data gathering and focus groups to get insight, other CX pros just bite the bullet and spend hours reading customer comments, labeling them, and funneling them into an unwieldy spreadsheet. They’re understandably frustrated by how difficult it is to get actionable insight.

By using text and sentiment analytics, humans can get huge quantities of customer feedback sorted and analyzed at the push of a button. Better yet, computers don’t have bad days or lose focus.

Once organized with tags, your time is freed up to look at the themes and trends that arise from the noise, then create actionable strategies based on those insights.

Now you can jump straight into action and the interns can work on more interesting, valuable projects!

PRO TIP: To get high quality insights at the push of a button, algorithms need to be trained. Be sure your feedback management software vendor has a team that will work with your data to ensure you get valuable insight from the start. With more data and occasional human guidance, you’ll get better and faster insight over time.

5. Your CX program lacks a real-time issue detection system.

An important element to providing a good customer experience is making sure any issues are handled quickly and efficiently. If you can detect and address them before your customer has a real issue, your CX program has paid for itself.

One of the benefits of having text and sentiment analysis is that your data and insights are updated in real-time. This means you have a new issue detection system.

Source: CXInsight™ Dashboard

This works best for a more mature customer feedback program with an established baseline, or status quo. For example, you know that on any given day, in any given geographic region, about 10% of your comments are tagged with ‘out of stock’ as an issue. When you check in and see that in Texas, 25% of comments coming in are tagged ‘out of stock’, that raises a red flag. You can immediately dig into specifics, read through the verbatims, and send those comments to the right people for follow up before the issue blows out of proportion.

The CX dream of being proactive in solving issues can be achieved with the help of automated organization of qualitative feedback.

6. Your internal teams aren’t agreeing on CX priorities.

It’s a given that successful companies focus on customer needs and experiences. The question is: is everyone at your company seeing the same information in the same way? If not, you’re wasting time and costly resources with competing priorities, and it is definitely time to invest in tools to fix it.

By having your CX tech parse the text and sentiment of your 1K+ daily inputs of customer feedback, you can democratize the information and insights across every team at your company. And that will ensure team leaders can quickly align to address the right priorities. So product development and customer support will be on the same page, and features will get developed (or possibly de-bugged) to meet the most important needs of the customer.

How does that happen? Feedback from every customer touchpoint is analyzed, from in-product surveys to emails. In this example, support ticket subject lines are auto-categorized and everyone from support to service to product to the c-level can see what issues are hot items to address.

Source: CXInsight™ Dashboard

Looking at the text analytics, it quickly becomes apparent that 15% of the support tickets are related to bugs that need to be addressed. On the proactive front, product could also delve into comments tagged “feature request” and focus on user concerns about UX/UI.

7. You need to demonstrate the ROI of your CX program.

Companies are eager to hop on the CX bandwagon, but it can still be a fight to get the proper resources to make a CX program thrive. You’ve probably already shown the C-suite the correlation between CX and revenue growth, but there’s pressure to squeeze a little more ROI out of what you’ve established.

Investing in a tool that pulls ROI from data is an expense. But it’s a more strategic spend than, and offers more immediate follow-up and action, than performing passive data review and organization. It’s also a moredirect value-add and much less expensive than hiring a third party human operation.

The cascading effects throughout the organization will increase ROI in the long-term as well.

Product teams can prioritize and build with evidence-based confidence.
Marketing teams will gain an understanding of different personas and see customers excited to spread the word about your business.
Support and operations teams will have early warning of potential issues and have more context when dealing with problems.

In the end, qualitative data is crucial to extracting value out of CX initiatives. Having more data from engaged customers should not be an obstacle.

Is this the point?

Are you seeing any of these 7 signs when you look at your company’s CX program? If so, do a cost benefit analysis. Typically, once your program has matured, the cost of tools that create actionable insights out of customer feedback are far cheaper than the cost of misaligned resources and long delivery times. Text and sentiment analytics make the resources you put into CX initiatives efficient, and turn the large quantity of unstructured data into an advantage by mining insight that would otherwise sit in limbo. Move this tipping point in your favor.

(Editor’s Note: this post is an update of a 2018 post)

Back to Blog

How to Create Meaningful Customer Experiences—Not Just Transactions

By: Andrew Park, VP, Customer Experience Strategy

October 27, 2020October 29, 2020

Conventional wisdom holds that customers shop the brands whose products and services best match their needs. But there’s more to the story than that. Even if it’s just a quick trip to the grocery store, customers seek something more profound from brands than a mere product: meaningful customer experiences.

There’s a lot for organizations to gain by orienting themselves around customers’ search for meaning. Experience programs can help them get there.

We’re going to go over exactly how companies can achieve that reorientation, create meaningful experiences for customers, and, ultimately, ride that heightened connectivity to the top of their respective verticals.

Right Audience, Right Problem

We touched on this in our last conversation about the importance of carefully designing your program before deploying it, but it’s worth saying again:

Some audiences are more worth brands’ time than others.

Sounds harsh, but let me explain. Some audiences offer context and solutions to problems that other groups may not even be aware of. Therefore, one of the first things brands should do to create meaning for their customers is consider the problems that can be solved by focusing on specific audiences.

This approach is vital is because it allows brands to hone in on customers’ “moment of truth.” This is the moment in which a customer finds significance in their interaction with a brand, not just a product or service.

What is preventing customers from finding their moment of truth? The answer to this question will dictate what you should design your listening program around.

Furthermore, that search will allow your company to create fundamental human relationships with customers. And those relationships will create positive buzz, build lifetime loyalty, and result in a much stronger bottom line.

Sharing the Love

Thinking how certain audiences can help solve business challenges is important, but it’s not the only step brands must take. Once a company’s experience team finds moments of truth, they absolutely must share the news across the organization! This sharing process is often called data democratization.

I really can’t say enough how important it is to share customers’ moments of truth. First, socializing that data across the organization gives every employee a glimpse of how their role affects the customer.

Second, sharing this intel makes it easier for brands to identify moments that matter out of mountains of experience program data. Ultimately, brands that intentionally democratize data from the beginning get so much more from their listening than companies who fail to design their strategy.

Listening Empathetically

The final key to creating meaningful customer experiences is on that is often overlooked: empathy. Empathy is the key to understanding moments of truth and, ultimately, business success.

Catering to customers’ search for meaning is neither a program luxury nor a saying you put on a wall sign. It’s a strategy that builds transformational brand success and the meaningful, emotional relationships that can sustain it indefinitely.

I go into greater depth about the importance of designing your experience program before listening in my article on the subject, which you can read here. Thank you!

Back to Blog

Text Analytics & NLP in Healthcare: Applications & Use Cases

This article explores some new and emerging applications of text analytics and natural language processing (NLP) in healthcare. Each application demonstrates how HCPs and others use natural language processing to mine unstructured text-based healthcare data and then do something with the results.

Healthcare databases are growing exponentially, and text analytics and natural language processing (NLP) systems turn this data into value. Healthcare providers, pharmaceutical companies and biotechnology firms all use text analytics and NLP to improve patient outcomes, streamline operations and manage regulatory compliance.

In order, we’ll talk about:

Sources of healthcare data and how much is out there
Improving customer care while reducing Medical Information Department costs
Hearing how people really talk about and experience ADHD
Facilitating value-based care models by demonstrating real-world outcomes
Guiding communications between pharmaceutical companies and patients
Even more applications of text analytics and natural language processing in healthcare
Some more things to think about, including major ethical concerns

NLP in the Healthcare Industry: Sources of Data for Text Mining

Patient health records, order entries, and physician notes aren’t the only sources of data in healthcare. In fact, 26 million people have already added their genetic information to commercial databases through take-home kits. And wearable devices have opened new floodgates of consumer health data. All told, Emerj lists 7 healthcare data sources that, especially when taken together, form a veritable goldmine of healthcare data:

1. The Internet of Things (IoT) think FitBit data)

2. Electronic Medical Records (EMR)/Electronic Health Records (EHR) (classic)

3. Insurance Providers (claims from private and government payers)

4. Other Clinical Data (including computerized physician order entries, physician notes, medical imaging records, and more)

5. Opt-In Genome and Research Registries

6. Social Media (tweets, Facebook comments, message boards, etc.)

7. Web Knowledge (emergency care data, news feeds, and medical journals)

Just how much health data is there from these sources? More than 2,314 exabytes by 2020, says BIS Research. For reference, just 1 exabyte is 10^9 gigabytes. Or, written out, 1EB=1,000,000,000GB. That’s a lot of GB.

But adding to the ocean of healthcare data doesn’t do much if you’re not actually using it. And many experts agree that utilization of this data is… underwhelming. So let’s talk about text analytics and NLP in the health industry, particularly focusing on new and emerging applications of the technology.

Improving Customer Care While Reducing Medical Information Department Costs

Every physician knows how annoying it can be to get a drug-maker to give them a straight, clear answer. Many patients know it, too. For the rest of us, here’s how it works:

You (a physician, patient or media person) call into a biotechnology or pharmaceutical company’s Medical Information Department (MID)
Your call is routed to the MID contact center
MID operators reference all available documentation to provide an answer, or punt your question to a full clinician

Simple in theory, sure. Unfortunately, the pharma/biotech business is complicated. Biogen, for example, develops therapies for people living with serious neurological and neurodegenerative diseases. When you call into their MID to ask a question, Biogen’s operators are there to answer your inquiry. Naturally, you expect a quick, clear answer. At Biogen Japan, any call that lasts more than 1 minute is automatically escalated to an expensive second-line medical directors. Before, Biogen struggled with a high number of calls being escalated because their MID agents spent too long parsing through FAQs, product information brochures, and other resources.

Today, Biogen uses text analytics (and some other technologies) to answer these questions more quickly, thereby improving customer care while reducing their MID operating costs. When you call into their MID, operators use a Lexalytics-built search application that combines natural language processing and machine learning to immediately suggest best-fit answers and related resources to people’s inquiries. MID operators can type in keywords or exact questions and get what they need in seconds. (The system looks like this illustration.) Early testing already shows faster answers and fewer calls sent to medical directors, and the application also helps new hires work at the level of experienced operators, further reducing costs.

Hearing How People Really Talk About and Experience ADHD

The human brain is terribly complicated, and two people may experience the same condition in vastly different ways. This is especially true of conditions like Attention Deficit Hyperactivity Disorder (ADHD). In order to optimize treatment, physicians need to understand exactly how their individual patients experience it. But people often tell their doctor one thing, and then turn around and tell their friends and family something else entirely.

A Lexalytics (an InMoment company) data scientist used our text analytics and natural language processing to analyze data from Reddit, multiple ADHD blogs, news websites, and scientific papers sourced from the PubMed and HubMed databases. Based on the output, they modeled the conversations to show how people talk about ADHD in their own words.

The results showed stark differences in how people talk about ADHD in research papers, on the news, in Reddit comments and on ADHD blogs. Although our analysis was fairly basic, our methods show how using text analytics in this way can help healthcare organizations connect with their patients and develop personalized treatment plans.

Facilitating Value-Based Care Models by Demonstrating Real-World Outcomes

Our analysis of conversations surrounding ADHD is just one example in the large field of text analytics in healthcare. Everyone involved in the healthcare value chain, including HCPs, drug manufacturers, and insurance companies are using text analytics as part of the drive towards value-based care models.

Within the value-based care model, and outcome-based care in general, providers and payers all want to demonstrate that their patients are experiencing positive outcomes after they leave the clinical setting. To do this, more and more stakeholders are using text analytics systems to analyze social media posts, patient comments, and other sources of unstructured patient feedback. These insights help HCPs and others identify positive outcomes to highlight and negative outcomes to follow-up with.

Some HCPs even use text analytics to compare what patients say to their doctors, versus what they say to their friends, to identify how they can improve patient-clinician communication. In fact, the larger trend here almost exactly follows the push in more retail-focused industries towards data-driven Voice of Customer: using technology to understand how people talk about and experience products and services, in their own words.

Guiding Communications Between Pharmaceutical Companies and Patients

Pharmaceutical marketing teams face countless challenges. These include growing market share, demonstrating product value, increasing patient adherence and improving buy-in from healthcare professionals. Lexalytics customer AlternativesPharma helped those professionals by providing useful market insights and effective recommendations.

Before, companies like AlternativesPharma relied on basic customer surveys and some other quantitative data sources to create their recommendations. Using our text analytics and natural language processing, however, AlternativesPharma was able to categorize large quantities of qualitative, unstructured patient comments into “thematic maps.” The output of their analyses led to research publications at the 2015 Nephrology Professional Congress and in the Journal Néphrologie et Thérapeutiques.

Further, AlternativesPharma helped customers verify assumptions made by Key Opinion Leaders (KOLs) regarding the psychology of patients with schizophrenia. This theory was then documented in collateral and widely communicated to physicians. (Full case study)

More Applications of Text Analytics and Natural Language Processing in Healthcare

The above applications of text analytics in healthcare are just the tip of the iceberg. McKinsey has identified several more applications of NLP in healthcare, under the umbrellas of “Administrative cost reduction” and “Medical value creation”. Their detailed infographic is a good explainer. Click the image (or this link) to read the full infographic on McKinsey’s website.

Meanwhile, this 2018 paper in The University of Western Ontario Medical Journal titled “The promise of natural language processing in healthcare” dives into how and where NLP is improving healthcare. The authors, Rohin Attrey and Alexander Levitt, divide healthcare NLP applications into four categories. These cover NLP for:

Patients – including teletriage services, where NLP-powered chatbots could free up nurses and physicians
Physicians – where a computerized clinical decision support system using NLP has already demonstrated value in alerting clinicians to consider Kawasaki disease in emergency presentations
Researchers – where NLP helps enable, empower and accelerate qualitative studies across a number of vectors
Healthcare Management – where patient experience management is brought into the 21st-century by NLP used on qualitative data sources

Next, researchers from Sant Baba Bhag Singh University (former link) explored how healthcare groups can use sentiment analysis. The authors concluded that using sentiment analysis to examine social media data is an effective way for HCPs to improve treatments and patient services by understanding how patients talk about their Type-1 and Type-2 Diabetes treatments, drugs, and diet practices.

Finally, market research firm Emerj has written up a number of NLP applications for hospitals and other HCPs, including systems from IQVIA, 3M, Amazon and Nuance Communications. These applications include improving compliance with industry standards and regulations; accelerating and improving medical coding processes; building clinical study cohorts; and speech recognition and speech-to-text for doctors and healthcare providers.

Some More Things to Consider: Data Ethics, AI Fails, and Algorithmic Bias

If you’re thinking about building or buying any data analytics system for use in a healthcare or biopharma environment, here are some more things you should be aware of and take into account. All of these are especially relevant for text analytics in healthcare.

First: According to a study from the University of California Berkeley, advances in artificial intelligence (AI) have rendered the privacy standards set by the Health Insurance Portability and Accountability Act of 1996 (HIPAA) obsolete. We investigated and found some alarming data privacy and ethics concerns surrounding AI in healthcare.

Read – AI in Healthcare: Data Privacy and Ethics Concerns

Second: Companies with regulatory compliance burdens are flocking to AI for time savings and cost reductions. But costly failures of large-scale AI systems are also making companies more wary of investing millions into big projects with vague promises of future returns. How can AI deliver real value in the regulatory compliance space? We wrote a white paper on this very subject.

Read – A Better Approach to AI for Regulatory Compliance

Third: The “moonshot” attitude of big tech companies comes with huge risk for the customer. And no AI project tells the story of large-scale AI failure quite like Watson for Oncology. In 2013, IBM partnered with The University of Texas MD Anderson Cancer Center to develop a new “Oncology Expert Advisor” system. The goal? Nothing less than to cure cancer. The result? “This product is a piece of sh–.”

Read – Stories of AI Failure and How to Avoid Similar AI Fails

Fourth: “Bias in AI” refers to situations where machine learning-based data analytics systems discriminate against particular groups of people. Algorithmic bias in healthcare AI systems manifests when data scientists building machine learning models for healthcare-related use cases train their algorithms on biased data from the start. Societal biases manifest when the output or usage of an AI-based healthcare system reinforces societal biases and discriminatory practices.

Read – Bias in AI and Machine Learning: Sources and Solutions

Improve Your Understanding: What Are Text Analytics and Natural Language Processing?

In order to put any tool to good use, you need to have some basic understanding of what it is and how it works. This is equally true of text analytics and natural language processing. So, what are they?

Text analytics and natural language processing are technologies for transforming unstructured data (i.e. free text) into structured data and insights (i.e. dashboards, spreadsheets and databases). Text analytics refers to breaking apart text documents into their component parts. Natural language processing then analyzes those parts to understand the entities, topics, opinions, and intentions within.

The 7 basic functions of text analytics are:

Language Identification
Tokenization
Sentence Breaking
Part of Speech Tagging
Chunking
Syntax Parsing
Sentence Chaining

Natural language processing features include:

Sentiment analysis

Entity recognition

Categorization (topics and themes)

Intention detection

Summarization

Lexalytics’ text analytics and NLP technology stack, showing the layers of processing each text document goes through to be transformed into structured data.

Beyond the basics, semi-structured data parsing is used to identify and extract data from medical, legal and financial documents, such as patient records and Medicaid code updates. Machine learning improves core text analytics and natural language processing functions and features. And machine learning micromodels can solve unique challenges in individual datasets while reducing the costs of sourcing and annotating training data.

Back to Blog

Wootric ranks #1 in ROI in Experience Management | G2 Grid Report

Note: Wootic was acquired by InMoment in January 2021. The Wootric product lives on as our Professional Plan.

Wootric, the CX management platform for maximizing customer lifetime value (CLV), has been recognized as a High Performer in the G2 Crowd Grid Report for Experience Management for Fall 2020 and Winter 2021. Wootric also outperforms the category on all satisfaction measures including ease of use.

Notably, Wootric, which seeks to drive business outcomes from customer experience efforts, has the fastest payback in the category.

Wootric is ranked #1 in ROI (Return on Investment)

“Companies should expect a financial outcome for their investment in CX,” said Jessica Pfeifer, Chief Customer Officer of Wootric, “Our turnkey approach means that our customers quickly understand user sentiment at the moments that matter, and analytics surface ways to immediately improve retention and engagement. It is gratifying to see our customers’ success reflected in our ranking.” In the G2 report, Wootric averages 9 months to return on investment, versus an average of 19 months for others in the experience management category, including Qualtrics and Medallia.

Experience management platforms help businesses bridge the gap between the experiences they believe they are delivering to customers and the experiences customers are actually receiving. They enable organizations to collect feedback from their customers with surveys that measure net promoter scores (NPS), customer satisfaction (CSAT), and customer effort scores (CES). By combining and analyzing customer feedback from multiple channels, experience management software offers companies a holistic view of their customers’ experiences and how those experiences are impacting the business.

Wootric specializes in customer experience management for high growth B2B and B2C software-as-service and companies in digital transformation. Over 1200 brands worldwide are understanding and improving the post-acquisition customer journey with Wootric’s CLV-focused approach.

Enterprise users also gave Wootric the top rank for usability and easiest admin

“We understand that in order to have an impact, CX champions must engage stakeholders, democratize insights, and ensure data is at the fingertips of frontline teams in real-time,” says Prabhat Jha, CTO of Wootric. “Their needs drive our roadmap — whether we are talking about our native integrations with modern tech stack players like Salesforce, Intercom, Hubspot, and Segment or our flexible Voice of Customer analytics hub that can be customized to meet the needs of numerous stakeholder teams like Product, Support, Success, and Customer Insights.”

The G2 Crowd Grid Report for Experience Management (Fall 2020) is a quarterly report that shows how the leading customer experience management solutions stack up to one another based on customer satisfaction and market presence. G2 Crowd’s scoring methodology blends data from user reviews and a vendor’s market presence, taking into account their social impact and market share, to generate the results for their Grid Report. Once scored, a vendor falls into one of four categories: Leader, High Performer, Contender, or Niche player.

This robust piece of research material should be read by CX practitioners and anyone evaluating a customer experience management solution. Learn exactly how each of the 26 companies included in the report received their score, their highest and lowest-rated features, satisfaction ratings, and more. Sign in and view this research on G2 to see:

Additional Data. Compare payback period (ROI) by vendor.
The G2 Crowd Grid visual
The Grid scores that determined each vendor’s placement
Side-by-side feature comparison
Methodology behind the scoring process

Learn how Wootric can help you improve customer lifetime value. Book a consultative demo today.

Back to Blog

The Case for Moving Your Experience Program Beyond Metrics

By: The InMoment Team

October 6, 2020April 15, 2021

For a lot of companies, the phrase “experience programs” brings careful management and lots of metrics to mind. Both of those things are important components of any experience effort, but they can’t bring about meaningful change and improvement. Experience programs can revolve around so much more than scoreboard-watching and reacting to challenges only as they arise—we’re going to go over how much more these programs can be and why brands should adjust their ambitions accordingly.

Movement Over Metrics

Conventional wisdom holds that if an experience program is returning great measurements, that must mean it’s really working for a brand. However, this isn’t necessarily true. Metrics are effective for highlighting a brand’s high points and weak spots, but that’s about it. A true experience program’s job doesn’t end with better metrics—that’s actually where the work begins.

Companies can create a fundamentally better experience for their customers (and thus a stronger bottom line for themselves) by taking action on their program’s findings. This means sharing intelligence throughout an organization rather than leaving it siloed, as well as encouraging all stakeholders to own their part of the process. In short, taking action is what makes the difference between being really good at watching scores roll in and actually fixing problems that might be muddying up the customer journey.

Narratives Over Numbers

The phrase “program findings” from the preceding paragraph can also mean more than just numbers. It can also denote customer stories, employee reports, and other, more abstract forms of feedback. Many experience programs pick this information up as a matter of course, but it can be difficult to take action on that intel without a concrete action plan.

One reason why many companies encounter this difficulty is because their programs don’t acknowledge a simple truth: some customer segments are worth more to listen to than others. It doesn’t make much sense to try to listen to every segment for feedback on a loyalty program that only long-term customers use or know about. This is why it’s important for brands to consider which audiences they want to gather feedback from before even turning any listening posts on.

Once brands have matched the audiences they want to listen to to the goals they want to achieve, that’s when they can turn their ears on and start gathering that feedback. Companies that take this approach will find feedback significantly more relevant (and helpful) than intelligence gathered through a more catchall approach. They can then perform a key driver analysis on those customers and put their feedback against a backdrop of operational and financial data for further context, which goes a long way toward the goal of all of this: meaningful improvement.

Experience Improvement Over Experience Management

Experience improvement is not a goal that can be reached just by reading metrics. It demands more than turning listening posts on and hoping that a good piece of customer intel comes down the wire. Rather, experience improvement demands action. Much like water molecules, the forces that drive customer expectations, acquisition, churn, and other factors are in constant motion, and thus demand constant action to stay on top of it all.

Desiloing intelligence, motivating stakeholders, and expanding program awareness to customer stories instead of just higher scores and stats is what makes the difference between an industry-leading experience and everyone else’s. These actions create better experiences for customers, compel employees to become more invested in providing those experiences, and creates a marketplace-changing impact for the brand.

Click here to learn more about how to take your program from simple metric-watching to meaningful improvement for all.

Back to Blog

AI In Financial Services: Three Current And Emerging Applications

While the impact of artificial intelligence (AI) is a bit of a mixed bag in a number of industries, we’re seeing some exciting traction in financial services. In this month’s article, I take a look at some specific examples of where machine learning and AI are helping financial services organizations improve their services, products, and processes.

AI Helps Financial Services Reduce Non-Disclosure Risk

Financial firms and banks are taking advantage of AI to ensure that their employees are meeting complex disclosure requirements.

Generally, financial advisors must make sure that their “client advice” documents include proper disclosures to demonstrate that they’re working in their client’s best interests. These disclosures may cover conflicts of interest, commission structure, cost of credit, own-product recommendations and more. For example, advisors must clearly disclose the fact that they’re encouraging a client to purchase a position in a company that the firm represents (a potential conflict of interest).

To ensure compliance, firm auditors randomly sample these documents and spot-check them by keyword or phrase searches. But this process is clunky and unreliable, and the cost of failure is high: Some estimates put the price of non-compliance as high as $39.22 million in lost revenue, business disruption, productivity loss and penalties.

To help financial services firms ensure disclosure compliance, companies like FINRA Technology, Quantiply and my company offer AI solutions that use semi-structured data parsing to analyze client advice documents and extract all of the component pieces of the document (including disclosures). Then, using natural language processing to understand the meaning of the underlying text, the AI structures this data into an easily-reviewable form (like an Excel document) where human auditors can quickly evaluate whether all necessary disclosures were made. Where before an auditor might spend hours to review 1% of their firm’s documents, AI solutions like this empower the same person to review more documents in less time.

AI Fights Elder Financial Exploitation

$1.7 billion. That’s the value of suspicious activities targeting the elderly, as reported by financial institutions in 2017 alone. In total, the United States Consumer Financial Protection Bureau (CFPB) says that older adults have lost $6 billion to exploitation since 2013. One-third of these people were aged 80 or older, some of whom lost more than $100,000.

Thankfully, tech companies and financial institutions are fighting back. The CFPB notes that “Regularly studying the trends, patterns and issues in EFE SARs [Elder Financial Exploitation Suspicious Activity Reports] can help stakeholders enhance protections through independent and collaborative work.” This is a great opportunity for machine learning and AI, which use reams of historical data to predict what is likely to happen next.

Wells Fargo, for example, uses machine learning and AI to identify suspicious transactions that merit further investigation. Ron Long, director of elder client initiatives for Wells Fargo Advisors, told American Banker earlier this year that their data scientists are constantly working to add new unstructured and structured data sources to improve their capabilities. “While a tool can’t replace human assessment,” he said, “machine-learning capabilities play an important part in our strategy to reduce the number of matters requiring a closer look so we can focus on actual cases of financial abuse.”

One example is EverSafe, an identity protection technology company founded in 2012, which draws on multiple data sources to train its AI. EverSafe places itself at the nexus of a user’s entire financial life, analyzing behavior across multiple accounts and financial advisors. This approach dramatically improves their AI’s ability to identify erratic activity or anomalous transactions. Eversafe’s founder, Howard Tischler, says he was inspired to create the company after his aging, legally blind mother was scammed multiple times, including by someone who sold her a deluxe auto club membership.

AI Adds A Crucial Competitive Edge In High-Frequency Trading

Back in the 1980s, Bloomberg built the first computer system for real-time financial trading. A decade later, computer-based high-frequency trading (HFT) had transformed professional investing. Some estimates put HFT at 1,000x faster than human-human trading. But since the 2010s, when trading speeds reached nanoseconds, industry leaders have been looking for a new competitive edge.

To keep up with (and ahead of) the competition, industry leaders are turning to algorithmic trading. The sheer volume of trading information available for machines to analyze makes artificial intelligence and machine learning formidable tools in financial marketplaces. Investment firms use AI to increase the predictive power of the neural networks that determine optimal portfolio allocation for different types of securities. In simpler terms: Data scientists use reams of historical prices to train computers to predict future price fluctuations.

AI has already proven its value in HFT. Renaissance Technologies, an early adopter of AI, boasted a return of 71.8% annually from 1994 to 2014 on its Medallion Fund (paywall). Domeyard, a hedge fund, uses machine learning to parse 300 million data points in the New York Stock Exchange, just in the opening hour. And PanAgora, a Boston-based quant fund, deployed a specialized NLP algorithm to quickly decipher the cyber-slang that Chinese investors use on social media to get around government censorship. These findings give PanAgora, a firm that operates at the speed of fiber optic cables, vital insights into investor sentiment fast enough to keep up with (and influence) its trading algorithms.

Wrapping Up: Tempering Expectations For AI In Financial Services

The value of AI in financial services is clear. But don’t get lost in the hype. For every useful AI system, you can find a dozen problematic algorithms and large-scale failures. To succeed, keep a realistic perspective of what AI can and can’t do to help.

The truth is that artificial intelligence is just a tool. Alone, AI doesn’t really “do” anything. What matters is how you combine AI with other technologies to solve a specific business problem.

This post originally appeared in Forbes Technology Council.

Back to Blog

Stop Managing Experiences—Start Improving Them

By: The InMoment Team

September 22, 2020September 22, 2020

InMoment® today announced its mission to challenge the customer experience industry and offer an elevated approach focused on Experience Improvement (XI)™ for the world’s customers, employees, and top brands. This involves dramatically increasing the results from experience programs through a new class of software and services specifically designed to help leaders detect and ‘own’ the important moments in customer and employee journeys. Read more in the full press release here.

Back to Blog

6 Competencies to Improve Customer Experience and Drive Business Growth

In the modern consumer-led environment, the customer experience is of paramount importance. Whether you’re offering omnichannel contact center solutions like we do at RingCentral or work in online retail, you must go the extra mile to ensure that your customers enjoy the best possible client experience. Because if you don’t improve customer experience, you can rest assured that your competitors will.

Where once the emphasis might have been on clever publicity schemes or B2B affiliate marketing, now businesses recognize that these methods – though they remain hugely important and valuable—need to be complemented by a laser-like focus on customer experience. Without that focus, you’ll find that consumers explore other options, and take their business where they feel it’s more valued.

Delivering exceptional customer experiences requires a wholehearted commitment across your company. That’s from the most senior management at the top, to the staff dealing directly with customers on the shop floor (or in the call center). It also requires a willingness to ask awkward questions about your own business, and a preparedness to confront – and address – any failings you might identify.

To improve customer experience and make your client encounters the best they can possibly be, you need to ensure that each interaction a consumer has with your business is smooth, consistent, and straightforward. You must also attend to their needs and concerns. Whether you’re showing customers how to record a webinar or helping them find the right furniture for their home, the same fundamental principles apply.

With all this in mind, then, it should be apparent that customer experience is vitally important to scaling a business. But what particular customer experience competencies can boost your business’s growth? This is what we’ll discuss in this guide, but first, we’ll look in more detail at exactly why the customer experience is of such overwhelming importance.

Furthermore, it is important to note that these expectations have changed quite dramatically in a relatively short period of time. The sheer choice available online has made it easier for consumers to shop around and explore alternative options. It’s no longer a matter of competing with businesses in your local vicinity, as there is (almost literally) a world of options out there for people to choose from.

Offering an exceptional customer experience, then, can have all sorts of welcome consequences. As we’ve discussed, it’s essential to ensuring long-term customer loyalty, thereby putting your whole business on a firmer footing. In addition, it also encourages positive word of mouth. Customers promoting your business to their friends and colleagues, and thereby (hopefully) bringing more custom your way.

The most important thing to remember with regard to customer experience is that the power these days lies with the consumer. No longer is it possible to pull the wool over their eyes with sub-standard products and services. The bar has been substantially raised, and this is the reality you must adapt yourself to.

Understanding Customer Expectations

Understanding customer expectations is crucial for businesses seeking to improve customer experience. It begins with identifying customer needs and desires, diving deep into what truly matters to them. By actively listening and gathering feedback, businesses can uncover valuable insights to tailor their products and services accordingly. Recognizing key touchpoints, such as initial contact, purchase, and post-sales support, allows businesses to focus their efforts and resources where it matters most. Mapping the customer journey provides a holistic view of the entire customer experience, enabling businesses to identify pain points and opportunities for improvement. By thoroughly understanding customer expectations, businesses can deliver personalized and exceptional experiences that leave a lasting positive impression.

Improve Customer Experience: 6 Essential Competencies

So, now that we’ve clarified just why customer experience is so vital, we need to discuss the competencies that can transform your business for the better and send its growth soaring into the stratosphere, as well as creating a healthier business environment for everybody involved.

Here are five customer experience competencies which you must be continually focused upon:

1) Prioritize the Customer

It might seem like an obvious point to make, but to deliver the best possible customer experience, you need to genuinely prioritize the needs of the customer. This is of the utmost importance whether you’re promoting a video hosting platform or trying to tempt people to try a new restaurant.

The exact experiences that people expect differ from sector to sector, of course, but prioritizing the customer must remain constant across the board.

Customers need to feel that they are the focus of your attention at all times. They need to feel valued at all times. This needs to involve more than just rhetoric or platitudes. Your plan to improve customer experience must be backed with a plan for action, including the resources necessary to back it up. Set clear metrics and use hard data to measure customer experience management.

2) Be Prepared to Engage

Another point that needs to be kept in mind is that customer engagement is an integral part of delivering exceptional experiences. Consumers these days love to give their opinions, and most of the time they don’t require much prompting to do so. This is something your business has to be properly prepared for. Customer engagement, therefore, needs to be a leading priority for your business.

It’s best to take the initiative yourself. Be proactive in your efforts to seek customers’ opinions, and provide your own outlets where clients can offer them. Of course, people will use social media too, so be alert to any discussion of your business there. Also, when customers provide feedback, listen to it. They may highlight issues of which you were previously unaware, and it’s always frustrating for customers to feel their views aren’t taken seriously.

3) Demonstrate Firm and Open Leadership

Much is often made of the importance of purposeful and strong leadership in business, and there’s little doubt that this can make a huge difference. The value of entrepreneurialism has been proven over many years, from the largest multinational corporations to the smallest mom-and-pop retail outlets. But when it comes to customer experience, business leaders themselves must be prepared to be led by customers.

Business leadership must, therefore, accept the necessity of customer-centric growth. This involves heeding the views of consumers and maintaining an intimate understanding of exactly what they’re looking for. This doesn’t mean following the whims of customers blindly, however. It’s a question of distinguishing the good ideas, ones that can genuinely advance your business, from the fly-by-night suggestions.

4) Get Employee Buy-in

Following on from the previous point, leadership isn’t about dictating from on high and then leaving everybody else to make sense of the latest diktat. If we’re working to improve customer experience, we need to make sure that an ethos of prioritizing the consumer’s needs saturates the whole business. This means that the entire team, from top to bottom, needs to buy into the idea.

Employees must be provided with the resources, tools, and training they need to provide customers with outstanding experiences. They could be helping a customer find the perfect holiday gift or undertaking a Google Analytics health check for a client. Either way, they need to be supported and encouraged in their efforts to enhance customer experiences.

5) Build Strong Brand Values

We’ve touched upon the importance of having a clear and compelling ethos. This also needs to inform the general values of your brand. You have to remember that customers will hold you accountable if your business, its products, and services don’t live up to the various values you espouse. Whether it be a commitment to attentive customer service, unbeatable value for money, or simply a pledge of consistent quality in all areas.

Also, it’s crucial that your values aren’t just hot air. They have to relate in a concrete way to what your business is doing. Your team must understand how the work they do ties into the broader values your business has articulated. It’s not enough to reel off some superficial platitudes and then try to present these as a mission statement. They have to actually mean something, and what’s more, they have to be seen to mean something by customers.

6) Designing Intuitive User Interfaces

Designing intuitive user interfaces is essential for creating a seamless and satisfying digital customer experience. One key aspect is simplifying website navigation, ensuring that users can effortlessly find the information or products they are looking for. By organizing content in a logical and user-friendly manner, businesses can reduce frustration and improve engagement, ultimately leading to higher conversion rates.

Optimizing mobile responsiveness is another crucial element in today’s mobile-centric world. With more users accessing websites through smartphones and tablets, it is imperative to provide a seamless and enjoyable mobile experience. Responsive design ensures that websites adapt to different screen sizes and resolutions, allowing users to easily navigate and interact with the content. By prioritizing mobile responsiveness, businesses can cater to the needs and preferences of their mobile users, enhancing their overall satisfaction.

Streamlining the checkout and conversion processes is vital for minimizing cart abandonment and maximizing conversions. Long and complex checkout procedures often deter users from completing their purchase. By simplifying the steps, minimizing form fields, and offering convenient payment options, businesses can create a frictionless experience that encourages users to convert. Streamlining the conversion process not only improves the user experience but also increases the likelihood of repeat purchases and fosters customer loyalty.

Measuring and Analyzing Customer Experience

Measuring and analyzing customer experience is vital for businesses to understand and enhance their interactions with customers. Key metrics serve as valuable indicators of customer satisfaction and loyalty. Metrics such as Net Promoter Score (NPS), Customer Satisfaction Score (CSAT), and Customer Effort Score (CES) provide quantitative insights into the overall customer experience. Collecting and analyzing customer feedback through surveys, reviews, and social media monitoring allows businesses to gain qualitative insights into customer perceptions and identify areas for improvement. By actively listening to customers, businesses can address their pain points and meet their expectations more effectively. Additionally, utilizing customer journey analytics provides a holistic view of the customer’s end-to-end experience across multiple touchpoints. This helps businesses identify moments of friction or delight along the customer journey, enabling them to optimize each stage and enhance the overall experience. By measuring and analyzing customer experience, businesses can make data-driven decisions, implement targeted improvements, and ultimately deliver a more personalized and satisfying experience that fosters customer loyalty and drives business growth.

Conclusion

It’s very easy to tell customers how important they are and how concerned you are to ensure their happiness. There’s scarcely a business in the world that doesn’t tell its customers how important they are, and how valued their opinions are, and so on. But far fewer businesses actually uphold standards like these. The crucial test for your business is to prove to customers that their welfare and satisfaction is your number one concern.

That said, if you can prove your sincerity, then improving the customer experience is the natural result. After all, when clients know that you are on their side, then they’re likely to be much more willing to work with you to optimize all aspects of your business.

In this guide, we’ve listed some of the core competencies which your business will need if it is to make these commitments into an everyday reality. Of course, the exact steps your business has to take to improve customer experience will vary according to a number of factors. They include the resources at its disposal, what competitors are offering, the sector it’s competing in, and so on.

You should also remember that your focus will inevitably change as your business prospers and expands. Nevertheless, you must redouble your efforts to ensure that your customers remain at the heart of what you do, and that you don’t lose sight of your original mission and purpose.

John Allen is Director, Global SEO at RingCentral, a global UCaaS, VoIP, and cti software provider. He has over 14 years of experience and an extensive background in building and optimizing digital marketing programs. RingCentral is a Wootric customer.

Back to Blog

Customer Lifetime Value: A Guide to the Northstar Revenue Metric

What Is Customer Lifetime Value?

The technical definition of Customer Lifetime Value (CLV) is the revenue earned from a single customer over time. It’s an equation that subtracts the cost to acquire a new customer (CAC) from the total revenue from that customer. The goal is to make the revenue-over-time from each individual customer as high as possible.

But the technical definition doesn’t cover the magic that’s actually in customer lifetime value – as a metric and as a mission for a digital marketplace, an e-commerce site, and SaaS businesses. Because when you go after customer lifetime value with intention, making it one of your “North Star” metrics, you’ll find that the cost-to-acquire actually shrinks. It becomes less expensive to acquire new customers, and the revenue pours in exponentially.

We are also at an inflection point with SaaS. While many SaaS companies are still largely concentrated on acquisition-based growth through demos and trials, we’re seeing a shift to focus on the end-user and the metrics that capture how happy they are, because those end-users lead growth. And that’s where customer lifetime value comes in as a business case. It is the ideal way to tie customer loyalty to revenue.

Those end users who are sticking with you are buying more from you (cross-sells and upsells) and they’re telling their friends and colleagues how great you are (referrals). In a sense, they become your virtual sales army. They’re out there warming up leads and sending them to you, so you don’t have to pay to find them.

This is the magic we’re going to unlock for you in this comprehensive article. If you want to know how to maximize your bottomline, then improving Customer Lifetime Value is key. And we’re going to explain how it all works, and how you can start using it to get better ROI for your business right now.

Part 1: Making the case for Customer Lifetime Value as the key metric for your customer experience strategy

I don’t know a single company that hasn’t pondered these questions:

What resource investment will have the most impact on customer health and revenue growth?
What can (or should) I automate?
Should I invest more money into customer experience (CX), customer support, or customer success right now?
Should we focus on building this new feature or should we focus on infrastructure improvements that might make our platform more secure or faster, etc.?
Should we invest in self-service onboarding to improve the journey for the end user?

The answers to all of these questions lie in Customer Lifetime Value.

If your business thrives on high-volume sales and high turnover, then you’re probably not a subscription-based business – but you also don’t need to worry so much about customer lifetime value.

But, if your business would benefit from high-volume sales AND returning customers AND lower acquisition costs, then customer lifetime value is your metric, and you’ve probably got your answers to the above questions. The more you invest in both user experience and customer experience, the less you have to invest in customer support, leading to organic growth and a higher customer lifetime value.

Customer lifetime value isn’t a passive metric – a numerical pat on the back for when you’ve done a “good job.” It’s an active, actionable metric that can be used in a few different ways.

Let’s look at a few different ways to use the Customer Lifetime Value metric:

CLV as Profit Metric

Traditionally, customer lifetime value has been used as a benchmark for whether your business is going well or going under. You look at your CLV/CAC ratio, and if it works out to at least 3 or higher (for every $1 dollar you spend acquiring a customer, you earn at least $3 dollars) you’re in the clear. You could then calculate the CLV/CAC ratio across your marketing channels to determine which are creating the most lifetime value (invest in those more) and which aren’t.

CLV as Customer Persona Builder

Once you start parsing out which clients have the highest customer lifetime value, you can look for what they have in common in terms of demographics, psychographics, user behavior, how they found you, and other characteristics. You can then use those commonalities to create better customer personas so you can go after higher CLV clients with intention.

Predictive CLV

Customer lifetime value can be used to predict the lifetime value of new customers when you examine current behavior and purchase patterns, and then base projected behavior and patterns based on those early indicators. You might already know how to predict churn based on “red flag” customer actions, and this concept is the same but in the opposite direction. You look for retention and upsell-predictive behaviors by reverse engineering what your best customers did at the beginning, middle, and ends of their journeys with you (if they’ve ended!).

CLV as Key Performance Indicator

Customer lifetime value is a broad KPI of how well you’re serving clients, how valuable your product or service is to them, and how well you’re delivering your solution with the appropriate customer experience. It’s a great North Star metric. You know you’re headed in the right direction as CLV rises. But, you’ll also need metrics that tell you, more granularly, what’s going on and why at each stage of the customer journey. So we also use Customer Journey Metrics like Net Promoter Score, Customer Effort Score, Customer Satisfaction, etc.

Once you start tracking customer lifetime value, you can a lot with it to improve your business – which we’ll get to in Part 3. But for now, let’s look at customer lifetime value as an equation – or really, several equations.

Part 2: Customer Lifetime Value as Equation – how to crack the code of calculating this complicated metric

If you are not mathematically-inclined, I’ll make this as straightforward as possible.

Customer lifetime value is revenue you expect to receive from a customer over time, less the cost of acquiring and keeping that customer.

Here it is in equation form:
CLV = (ARPU X average # of months or years retained) – (CAC + CRC)

People have been refining ways to calculate more accurate CLV ratios for years. What’s so hard? So. many. variables. Here are the basic numbers you’ll need for the CLV calculation for a SaaS business:

Average Monthly Revenue Per Customer (ARPU)

Here are all the different ways customers bring in value in a subscription software business model.

Original revenue
Renewal revenue
Upsell revenue
Cross-Sell revenue
Referral Revenue

Most calculations only deal with original revenue and renewal revenue, but that doesn’t cover the whole picture. When calculating Average Monthly Revenue Per Customer (ARPU) for our customer lifetime value equation, just remember to account for upsells and cross-sells, not just original revenue and renewal revenue. Referrals take care of themselves — they’ll show up in the customer acquisition cost (CAC) calculation because you’ll see that you’re getting more new customers without spending more on sales & marketing.

You’ll also want to know your CAC because the two are intertwined. Your CLV will increase if you are able to increase revenue from customers while maintaining or lowering your acquisition cost.

Customer Acquisition Cost (CAC)

How much you spent on sales & marketing in a given time period (learn more about this here)

Divided by…

How many new customers you gained in the same given time period

Customer Retention Cost (CRC)

The cost of serving the customer is often overlooked in CLV calculation. And if your onboarding customer success and/or customer service programs are significant, you definitely want to factor in Customer Retention Cost. Totango, a Wootric integration partner, wrote a whole book on calculating CRC, but a quick estimate looks like this:

How much you spend to onboard, train and support customers in a given period

Divided by…

How many new customers you gained in the same given time period

Customer lifetime value calculation in non-subscription models

One more way to calculate CLV is through a predictive model that can be highly accurate. This method is common in consumer businesses such as e-commerce. That equation looks like this:

CLV = (Average monthly transactions X Average order value) X Average gross margin X Average Customer Lifespan*

*The average customer lifespan is calculated in months.

Segment Customer Lifetime Value to make it more actionable

Calculating customer lifetime value for your company can be revealing and is a great start to working with this metric. Like measuring NPS though, it really isn’t actionable until you start segmenting the metric. To make customer lifetime value more actionable and predictive, you’ll want to separate these numbers by customer segment and acquisition channels. That’s when you’ll be able to optimize your acquisition strategies to raise your CLV rates even higher.

Start by looking at customer lifetime value by pricing tier or persona. For example, you may discover that the CLV for enterprise customers is no higher than self-service customers once you factor in the high cost of acquiring and supporting “the big fish.”

Part 3: The 4 Most Powerful Ways to use Customer Lifetime Value to grow your business

To use CLV as an actionable, predictive, productive metric, you have to segment your users and rank them by their CLV. Then you can look at the data you’ve collected on them – which acquisition channels they came from, what their first interaction was on your website, what their customer journey looked like through onboarding and beyond – to optimize each stage of the customer journey.

And then you can return to customer lifetime value as a ‘big picture’ measurement of your optimization progress.

Here are three primary ways to use customer lifetime value to optimize acquisition and retention.

1. Optimize your acquisition strategies for CLV – and use CLV to optimize your acquisition strategies.

Your CRM platform should tell you which channels customers came through to find you, and you may notice that your high-CLV customers tend to come from one of those channels over the others.

One of the most clear-cut stories of how a big company used customer lifetime value to increase profit is IBM. IBM used customer lifetime value to determine the effectiveness of their marketing channels to attract high-spending customers – direct mail, telesales, email, and catalogs per customer (yes, this is an old story – way back in 2008). When they reallocated resources to the best-performing channels, they 10Xed their revenue.

It’s low-hanging fruit to decide to spend more marketing money on the channels yielding the highest CLV clients. But we can go one step further.

2. You can use your Customer Lifetime Value to create better buyer personas.

Yes, this requires a platform that can gather all of the available information on each customer. But use whatever information you’ve got. You will find that your high CLV customers have a lot in common (though you may need to form segments for the commonalities to clearly emerge).

Once you have your high CLV buyer personas, you can use them to form marketing, outreach, and retention strategies based on their specific acquisition channels and user behavior through onboarding and retention.

For example, let’s say that you find that your high CLV clients come to you through G2 or Capterra. And once they reach your site, they don’t just “buy now” – they have at least one interaction with your live chat helpline. Your high CLV customers need a conversation before converting, which means if you tweak your G2 listing or website content answer their questions without having to reach out, you’ll likely see higher conversions from customers who’ll stick around.

3. Use Customer Lifetime Value with Customer Success for higher retention rates & referral revenue

Customer lifetime value and customer success are so intertwined as to be inseparable. Why? Successful customers don’t leave. So, when you want to improve your customer lifetime value, having a customer success program in place is one of the best ways to do it. Customer success asks, at each stage of the customer journey: What is the customer’s ideal outcome, and how can we best move them towards it? Then the customer success team can create strategies around supporting customers at pivotal moments – like places in the onboarding process where customers tend to get frustrated and leave (Customer Effort Score surveys are ideal for flagging these points of friction) or using churn-predictive behaviors to ‘red flag’ certain interactions to receive Customer Support pop-up chats.

4. Use Customer Lifetime Value to obtain more referrals from customers.

Your long-term high CLV customers are your brand ambassadors and influencers, and once you identify them, you can start to leverage that by rewarding and strengthening their connection to your brand. That could be something as simple as inviting them to be part of a free Beta testing group, so they can give you their insights into the next iterations of your product or service, or even just asking them to write an online review. Some businesses host online communities for their best clients, or offer them priority support.

Want an even easier way to identify the customers most likely to refer you to others? Learn how to use NPS surveys to not only find your promoters, but encourage them to promote you more.

5. Use Customer Lifetime Value to guide product design and validate product development decisions at the business level.

Product teams may be removed from revenue goals on the day-to-day, but strategic decisions about where to expend engineering resources should be made with business impact in mind. Product can use CLV to inform what customer segments the product should be designed for. Building CLV-related goals into user stories or feature specifications can help prioritize the roadmap and provide a success metric for retrospective once the product is out the door.

Part 4: 10 Ways to Increase & Optimize Your Customer Lifetime Value

Prioritize customer experience above everything else. And don’t just say it; measure it with metrics like Net Promoter Score, Customer Satisfaction, and Customer Effort Score. Calculate and track churn rates and engagement metrics.
Invest in customer success. Customer success drives acquisition, retention, and customer spending (upsells and cross-sells), raising customer lifetime value by helping customers achieve their ideal outcomes.
Invest in UX testing. The data you get from UX testing makes your product easier to use, reducing friction, and making it a must-have tool for your users.
Pay special attention to onboarding. Churn happens most frequently during or shortly after onboarding, so paying attention to churn-predictive behavior patterns (often identified by a Customer Effort Score survey) in the onboarding process can help you form a strategy to smooth those friction points and find easier ways to move your client towards meaningful success milestones.
Bring product management, customer success, customer support, and marketing together in shared responsibility for metrics. Collaboration between product and customer success is common, but it is a good idea to expand the team because they have so much to gain from working together. For example, with onboarding, product managers need to understand how their tech decisions affect adoption and retention metrics; and customer success teams need to have access to onboarding user data that helps them identify upsell opportunities. Some shared metrics for success include NPS, churn rate, trial conversion rate, adoption rate, and, of course, customer lifetime value.
Use CLV as a segmentation tool. This allows you to deliver appropriate experiences to customers who are high-value, and who have the potential to become high-value. The appropriate experience might be the level of customer support each segment receives, or the messaging they get throughout their buyer’s journeys. You may also find that each CLV segment has different pain points and needs, which you can target for even higher acquisition and retention rates.
Ask your most loyal customers for support. Following up a positive NPS survey response with an automated request for an online review is simply asking happy customers to follow through on what they just said they’d be willing to do. They’ve already said yes – so make it easy for them to act on promoting you. This won’t directly affect your CLV score, but it will drive down your CAC as the referrals come in.
Keep customers engaged by adding value to your product or service, or through high-value content. If you don’t have substantial updates/improvements/expansions planned for your product, you can keep customers engaged with educational materials–i.e. content that helps them reach their ideal outcomes faster and easier. This has the added benefit of being useful for top-of-funnel marketing as well.
Listen to your customers and act on their feedback. Voice of customer data is so important for improving products and reducing friction. The only problem is that sentiment analysis at scale can be difficult without the right tools.
Don’t “acquire customers” – build relationships. The customers who stay with you the longest feel like they know you. They feel like you know them. You’ve become an integral part of their daily lives, and they’d miss you if you went away. So consider changing the way you think of acquiring customers. You’re building relationships. And the more personalized and personal you make your customer interactions, the more likely your customers will feel connected to you and committed to your brand.

Maximizing Customer Lifetime Value is really a whole-company effort, requiring a great product, great service, and a deep understanding of your customers’ needs, frustrations, and desires. It’s a ‘big picture’ metric; a North Star number to guide you towards creating better customer experiences. But this one metric can also shed light on valuable segments and strategies that can profitably impact your business. customer lifetime value is a number you can’t afford to ignore.

Back to Blog

5 Guidelines for Structuring Your Product Roadmap

Building and maintaining the product roadmap is a central part of your role as a product manager. Yet there is surprisingly little consensus about product roadmaps across the product management community. Opinions vary wildly, for example, about what exactly a product roadmap is, how to structure one, what to include in it, and which tools you should use to develop it.

In this post I will offer a few guidelines for how to structure your product roadmap in ways that can lead to the development of a successful product. But before we dive into these suggestions, I would like to start with two fundamental points about roadmaps — points I hope will make the guidelines that follow much clearer.

The top-down approach to product development works best.

For successful product development, I recommend a top-down product strategy. Product roadmaps fit very strategically into this hierarchy. Here’s how it works.

Start with your product’s vision — which you’ll derive from your company’s larger strategic objectives. Then translate that high-level vision into actionable goals. Next, turn those product goals into your product roadmap. Finally, move from your roadmap to your backlog.

Starting at the highest level, and working your way strategically down into the details, is the best way to stay focused and on track toward your main objectives — and to avoid losing sight of why you’re doing what you’re doing and getting lost in the weeds.

And speaking of getting lost in the weeds…

A product roadmap is not a list of features.

A list of features is just that — a list of features. The product roadmap, on the other hand, is a strategic document that represents your high-level goals for the initiative along with an execution strategy to communicate how you plan to achieve those goals.

A roadmap might include features, of course, but the features themselves are only a part of the execution plan.

Five Guidelines for Structuring Your Product Roadmap

1. Product Roadmaps Should be Flexible

You’ve got to be okay with uncertainty. You’ve got to be willing to embark on your product plan, knowing that you can’t possibly know everything at the outset — that you will hit surprises, challenges, and opportunities along the way. Your roadmap needs to factor in these uncertainties — and needs to be flexible enough to allow you to change course quickly when necessary.

To avoid making promises you can’t keep, make sure your stakeholders understand that the roadmap is not an end-all, be-all document. One way to do so is to show more granularity in the short term while keeping your initiatives high-level and your dates approximate in the long term.

Thinking in themes is another effective way to keep your roadmap flexible. For example, maybe you and your stakeholders have agreed to focus on increasing conversions from a particular target persona in the upcoming quarter. This area of focus becomes your theme, and you retain the ability to reprioritize specific features and fixes within that theme while keeping the overarching goal intact. Likewise, if an opportunity arises that was not originally on your roadmap but furthers your shared goal, you’ll have an easier time making a case to include it.

Finally, a flexible roadmap can be a headache if changes are not communicated promptly and effectively. Ensure your stakeholders always have access to the most up-to-date version of the roadmap, whether that means storing it on a shared wiki or using cloud-based software that automatically updates. Transparency is key to building alignment on product strategy, no matter how frequently you need to reprioritize.

2. Product Roadmaps Should be Deeply Rooted in Well-Thought-Out Goals

This might sound like a contradiction to the previous guideline — but it’s not. You can frequently reprioritize your backlog while still staying true to well-thought-out, agreed-upon goals.

Your specific roadmap priorities might need adjusting in light of new information — in fact, they almost certainly will at one time or another. But you should be prepared to adjust focus in your roadmap, reallocate resources or otherwise change direction only if doing so is in alignment with your product’s high-level strategic goals and vision.

As a product manager, it’s your job to make sure that you’re making decisions about what to do, and what not to do, for the right reasons. Even if a sales executive is loudly demanding the immediate inclusion of a new feature to close a deal, you still need to weigh the request in light of your organization’s strategic goals. Quick wins do not necessarily spell long term success. If the feature isn’t in line with your product vision, it’s okay to say no. Indeed, if you’re doing your job well, you’ll find yourself saying no quite often.

3. Product Roadmaps Should be Developed with Plenty of Input

You don’t need to craft your roadmap yourself. You shouldn’t. Silos rarely work for any business function, and for roadmaps they can lead to ill-advised plans and products developed without critical knowledge.

To successfully bring a product to market, or to update an existing product in such a way that benefits the customer and your company, you’ll need plenty of input from experts across a variety of teams and departments. That includes, for example, engineering, customer support, sales, and marketing.

Sales, for example, will have valuable anecdotes about their most recent wins, or their most important ones. They can also tell you about the features prospects and customers are asking for.

As for your engineers, the more you involve them in the creation process, the more ownership and responsibility they will take for their role, and the more creativity and enthusiasm they’ll bring to the project. Collaborating with your engineers, and soliciting their help, will make them feel more like a part of the process, and less like order takers simply being told what to do by a product manager.

Prabhat Jha, CTO of the Net Promoter Score platform InMoment adds, “Be sure to solicit plenty of user input. Some years ago I learned this the hard way. The software enterprise where I worked did not have a rigorous system in place for listening to customers. We had collaborated internally on our roadmap priorities and thought we knew our customer’s needs. We missed key features, though, and as a result, there was very little adoption of the first version of the product. We ended up having to do a second release very quickly in order to get traction.”

4. Product Roadmaps Should be Visual

Ultimately, a product roadmap is a communication tool — an execution strategy you will use to convey your product plans and goals to a variety of constituencies. And the best way to communicate a complex initiative is visual. If your roadmap is simply a long list of features presented in a spreadsheet, people’s eyes will glaze over.

Visual roadmaps make it easy for everyone in your strategy meetings to quickly understand what you are proposing and hoping to accomplish quickly.

Another valuable reason to make your product roadmap visual is that it forces you to be ruthless about which initiatives to include and which to leave out. Venture capitalist Guy Kawasaki’s “10/20/30 Rule of PowerPoint” instructs that PowerPoint presentations should have 10 slides, last no longer than 20 minutes, and contain onscreen text no smaller than 30 points. In addition to making the presentation more digestible for an audience, this exercise forces the presenter to distill both the whole talk and each individual slide down to its most essential elements.

The same is true for product roadmaps. If you use a visual tool for creating your roadmaps — such as PowerPoint or a visual roadmap software application — that exercise will force you to distill your plan down to only those initiatives that serve your product’s strategic goals and vision.

Looking at this from another angle, if your roadmap contains 1,000 initiatives, that probably means you haven’t done a good job of prioritizing and strategizing what needs to be included in your product. Indeed, if you do have a list of 1,000 initiatives, your document could probably be more accurately called a backlog than a roadmap.

5. Tailor Your Roadmap Discussion to Your Audience

You are likely going to show your product roadmap to many different constituencies, in many different meetings, throughout the development cycle of your product. Each constituent group will have a unique focus and set of priorities, and each meeting will call for you to delve into different aspects of the product roadmap.

There are a couple of ways of accomplishing this. You can create separate roadmap documents for each constituency, or you can use a single, general roadmap and zoom in to what’s important for each group. The important thing is that you provide your stakeholders context and show them how your plans will further their unique goals.

For example, for your sales team, you might want to highlight the aspects of your roadmap that are designed to bring them better-qualified leads — for example, a trial version of your software that can help to prequalify interested prospects.

Customer-facing teams also likely had input into the features that made it onto your roadmap in the first place. Be sure to communicate to them where their requests fit into the plan (or didn’t) and why.

However, when presenting the roadmap to your executives, you’ll want to focus on how the choices you’ve made will lead to increased revenue or grow market share.

And when meeting with your engineering team, you’ll want to focus both on the high-level themes and specific feature details and discuss how your engineers can help to make those product goals a reality.

In other words, you want to keep your product roadmap flexible enough that it can help facilitate a productive meeting at any level of detail needed, with any constituency group you are presenting to.

Conclusion: Whatever structure your product roadmap takes, its main job is to communicate your strategy.

A product roadmap needs to communicate your strategy. It is your job to create your roadmap in such a way that it lays out a high-level execution plan for the product’s successful development and eventual launch into the market. It’s also your job to make sure all relevant constituencies understand the goals of the roadmap — and work with you to achieve them.

New things will always come up — cool ideas for new features, requests from executives for a shift in priority, urgent demands from sales reps, and so on. The challenge for the product manager is to view each one as an opportunity to evaluate through the lens of strategy and goals and help drive a sound decision-making process.

That’s why you need to start with your product vision, and from there derive specific goals — and only after you’ve developed those goals, build your product roadmap. If you haven’t fully fleshed out your vision and strategic goals for the product, it’s too early to start building your roadmap.

About the author:
Andre Theus is the Vice President of Marketing at ProductPlan. He works closely with customers and prospects to build better product roadmap software. Prior to ProductPlan, he was a member of marketing teams at RightScale, Sonos, and Citrix. Andre received a master’s in computer science from the Cologne University of Applied Science in Germany.

Originally published June 28 2016, updated July 28 2020

Start getting user feedback today with InMoment.

Back to Blog

The 80/20 NPS Guide for B2B SaaS

In this guest post, Nathan Lippi, Head of User Research at PandaDoc, shares a Pareto principle approach to getting the most from a B2B Net Promoter Score program.

NPS. It’s debated, loved, and hated, but in the world of B2B SaaS it’s rarely used to its full potential.

At PandaDoc, we’ve become increasingly customer-obsessed since the introduction of our NPS program two years ago, but we feel as if we still have meaningful room for improvement.

We’ve found there isn’t much written about NPS, specifically for B2B companies, so in order to level up, we’ve gone straight to the experts. With their permission, we’re sharing some key findings here.

We hope this guide helps you to get the most out of your CX program.

Let’s get to it!

The main purpose of NPS is to drive action

NPS is an easy, trusted, and benchmark-able way to start driving customer-focused action at your company.

Many companies obsess too much about the number when they’re starting out.

However, the most successful companies never lose sight of the fact that the primary purpose of CX metrics is to drive customer-focused action.

Once you’re driving customer-focused action, you’ll start to actually reap the benefits of increased retention, expansion, and word of mouth.

One Oracle VP’s Three-Step Recipe for NPS Survey Success

Joshua Rossman is an NPS OG, having run NPS at eBay and McAfee, among other companies. He’s now Vice President, Customer Experience Strategy at Oracle.

Through his years of experience, Rossman has created a three-step system he uses to get the most out of customer experience surveys, including NPS. He’s been kind enough to give us permission to share it publicly.

Step 1: Ask an easy-to-answer anchor question first to improve response rates

This principle is standard for NPS, but powerful enough to use across other CX surveys.

Ask your broad question first, and get a quantitative rating. Making your first question easy to answer will improve your overall survey response rates.

Step 2: Get S-P-E-C-I-F-I-C with your open-ended ask

Rossman has found that the standard open-ended question, “Care to tell us why?” often leads to vague, inactionable responses (e.g. “It’s hard to use”).

He’s found that asking promoters for specific reasons they recommend — and non-promoters for specific ways to improve — leads to much more actionable feedback.

Here are the specific questions he recommends for brand-level NPS:

Promoters: “What is it that makes you most likely to recommend {{company}}?”

Non-promoters: “What is it we could do that would make you more likely to recommend us in the future?”

These questions ask more specific questions — and tend to get more specific answers.

Various platforms such as InMoment can help you automatically categorize your now-more-specific NPS verbatims!

Each company will want to tag their work in a way that makes the most sense to them, but Shaun Clowes, former Head of Growth at Atlassian, says that they used machine learning to tag their feedback into three categories: Reliability, Usability, and Functionality. They used the ratio of different complaints to understand, at a high level, where their product needed work.

Step 3: NPS’ Secret Third Question

Even with the more specific responses you’ll hopefully get from the tweaks recommended in Step 2, not all B2B companies get such a high volume of responses that they can glean mathematically reliable responses from text alone.

One way to gain a deeper understanding of the factors that lead to an excellent (or poor) user experience is to follow questions about satisfaction with questions about various attributes of your brand. Ask a few extra questions with NPS and you can capture the overall sentiment for each area:

Wireframe example | Rating satisfaction of multiple attributes

After you’ve captured these details you can then run a simple linear regression, which will tell you which factors most influence if a person is a promoter or a detractor.

Various versions of the linear regression technique were also mentioned by Allison Dickin, VP of User Research at UserLeap, and other experts. Hearing them reinforce the power of this third question helps us get really excited about what we might do with it.

“Extra questions should be used judiciously,” counters Jessica Pfeifer, Chief Customer Officer at Wootric. “Think about it: When was the last time you responded thoughtfully to a multi-question survey?”

If you’re worried that such a long third step may lead to a negative user experience or lower response rates, a lighter option may be to ask the respondent to tell you what drove their score by selecting from a pick list of reasons.

If your follow-up question to detractors is, “What is the main thing we need to improve?,” you could offer a picklist that includes product, support, training, and value.

Not only are you learning what’s driving your score overall, but you’re also generating groups of users to follow up based on their interest. For example, your customer support team can learn more by reaching out to detractors who cite “support” as an issue.

Example 2-step in-app NPS survey with a pick list

Drive Strategic Action with a Cross-Functional Cadence

You may have noticed that our first heading was about driving action on behalf of the customer.

We’re touching on it again because, ultimately, driving action on behalf of your customers should be the primary concern of an NPS program.

Driving tactical action on behalf of customers was something we were already doing well at PandaDoc, before talking to the experts. Getting NPS data into Slack and other systems has been a pillar of our NPS program — this helps us take immediate action on issues that surface in feedback. One example: reaching out to an unhappy detractor and quickly fixing the issue that her NPS feedback brought to our attention.

However, learning how many companies drive strategic action on behalf of the customer in the following way, was eye-opening:

Collect customer feedback in a central repository (NPS, sales feedback, CS feedback, etc. — all combined together, somewhere like InMoment, UserVoice or ProductBoard.
Perform a 360° analysis of this data on a quarterly basis
Set up a monthly cross-functional cadence to decide which action to drive, and to track progress and accountability on ongoing courses of action

Fictional Examples of Driving Strategic Action:

Product Team

Diagnosis: Self-serve onboarding is our most common NPS complaint. People often come away without understanding our platform’s core concepts.

Initiative: Improve self-serve onboarding to teach core concepts of the platform.

Success Team

Diagnosis: Feedback about CS indicates all roles except admins are quite happy. Admins specifically have trouble understanding how to set user permissions, and they’d rather avoid going through training to learn something so small.

Initiative: Create micro-videos that explain to admins on how to manage user permissions.

Support Team

Diagnosis: NPS feedback indicates enterprise customers are unhappy with the time it takes to resolve support interactions involving custom features.

Initiative: Route tickets from enterprise customers directly to senior agents who have the expertise and product knowledge to resolve their issues.

Marketing

Diagnosis: Many of the leads we’re attracting cannot benefit from our core value proposition.

Initiative: Better align their SEM campaigns and landing pages with promises that the product can fulfill.

Your metrics should flow from your unique business strategy

NPS has been sold by some as the be-all / end-all metric of a customer-centricity program. But this approach can be harmful.

While NPS is often a great way to understand brand-level sentiment, it makes sense to layer on additional metrics as your CX program progresses.

Jessica Pfeifer at Wootric and Allison Dickin at UserLeap agree on the idea that your CX metrics should flow from what’s most critical to your business’ success.

“You’ll be able to benchmark and track trends over time when you complement NPS with established metrics like CSAT, PSAT, or Customer Effort Score at critical touchpoints in the customer journey,” says Pfeifer.

“For example, you might trigger a Customer Effort Score survey to gauge how easy it is for a user to achieve ‘first value.’ What is that critical milestone in your product? In PandaDoc’s case, it might be sending a document. Here at Wootric, it’s when a customer has live survey feedback flowing into their dashboard.”

Both took time to talk to us about questions that can be used in addition to (or as an alternative to) NPS. Here are some examples:

Example Non-NPS Questions

Business question	How to ask it
Examples from Allison Dickin @ UserLeap
What are the factors that affect churn, and what can we do differently to reduce it?	First question: How likely are you to use {{company}} for the next 3 months? Follow-up question: What would make you more likely to continue using {{company}}?
How well are we delivering on our core value proposition?	First question: How well does {{company}} meet your needs for {{value prop}}? Second question: How could {{company}} better meet your needs?
How is our first session going for users, and how can we improve it? One option here is to pop up a question in-app, before the median session time. Another option is to email users after their first session.	First question: How would you rate your experience getting started with {{company}}? Second question: How could {{company}} better meet your needs?
Examples from Jessica Pfiefer @ Wootric
How satisfied are users with our product, a feature, or service and how can we improve them? E.g. support interactions. Survey in product for feedback on features, survey via email or Intercom Messenger for support interactions.	CSAT First question: “How satisfied are you with your recent support interaction? Second question (customize based on score): “What could we do to improve?
We have a key but difficult task that we need to make easier for users. How difficult is the task, and how can we make it easier to do?	CES First question: “How easy was it for you to {{key but difficult task}}? Second question: “What could we do to improve?”

Takeaways

NPS is a great way to get started with driving customer-centric action
Use Josh Rossman’s three-part system to get the most out of your CX surveys, including NPS
Use analysis and a cross-functional cadence to drive org-wide, customer-focused action
As your business grows, layer on metrics that fit your specific business needs

This is just the tip of the iceberg for NPS, but we hope it will help your company squeeze the most out of your CX research program.

Hit me up on Twitter (@nathanlippi), and to let me know what’s worked well for you and your company!

Retain more customers with InMoment, the #1 Net Promoter Score platform for SaaS

2023 Consumer Experience Trends

2023 Consumer Experience Trends

A quick primer on AI training sets

Training model process

Manual tagging: an inconsistent truth

Tag definitions vary

Getting to a good training model

Ready for autocategorization

1. You have a mature or quickly-maturing CX program.

2. You receive 500+ comments per month (or you’re headed there.)

3. You’re sitting on a goldmine of feedback, but unable to get actionable insights.

4. Manual feedback organization & categorization is insightful, but painfully slow.

5. Your CX program lacks a real-time issue detection system.

6. Your internal teams aren’t agreeing on CX priorities.

7. You need to demonstrate the ROI of your CX program.

Is this the point?

Right Audience, Right Problem

Sharing the Love

Listening Empathetically

NLP in the Healthcare Industry: Sources of Data for Text Mining

Improving Customer Care While Reducing Medical Information Department Costs

Hearing How People Really Talk About and Experience ADHD

Facilitating Value-Based Care Models by Demonstrating Real-World Outcomes

Guiding Communications Between Pharmaceutical Companies and Patients

More Applications of Text Analytics and Natural Language Processing in Healthcare

Some More Things to Consider: Data Ethics, AI Fails, and Algorithmic Bias

Improve Your Understanding: What Are Text Analytics and Natural Language Processing?

Wootric is ranked #1 in ROI (Return on Investment)

Enterprise users also gave Wootric the top rank for usability and easiest admin

Movement Over Metrics

Narratives Over Numbers

Experience Improvement Over Experience Management

AI Helps Financial Services Reduce Non-Disclosure Risk

AI Fights Elder Financial Exploitation

AI Adds A Crucial Competitive Edge In High-Frequency Trading

Understanding Customer Expectations

Improve Customer Experience: 6 Essential Competencies

1) Prioritize the Customer

2) Be Prepared to Engage

3) Demonstrate Firm and Open Leadership

4) Get Employee Buy-in

5) Build Strong Brand Values

6) Designing Intuitive User Interfaces

Measuring and Analyzing Customer Experience

Conclusion

What Is Customer Lifetime Value?

Part 1: Making the case for Customer Lifetime Value as the key metric for your customer experience strategy

Let’s look at a few different ways to use the Customer Lifetime Value metric:

Part 2: Customer Lifetime Value as Equation – how to crack the code of calculating this complicated metric

Part 3: The 4 Most Powerful Ways to use Customer Lifetime Value to grow your business

Part 4: 10 Ways to Increase & Optimize Your Customer Lifetime Value

The top-down approach to product development works best.

A product roadmap is not a list of features.

Five Guidelines for Structuring Your Product Roadmap

1. Product Roadmaps Should be Flexible

2. Product Roadmaps Should be Deeply Rooted in Well-Thought-Out Goals

3. Product Roadmaps Should be Developed with Plenty of Input

4. Product Roadmaps Should be Visual

5. Tailor Your Roadmap Discussion to Your Audience

Conclusion: Whatever structure your product roadmap takes, its main job is to communicate your strategy.

The main purpose of NPS is to drive action

One Oracle VP’s Three-Step Recipe for NPS Survey Success

Step 1: Ask an easy-to-answer anchor question first to improve response rates

Step 2: Get S-P-E-C-I-F-I-C with your open-ended ask

Step 3: NPS’ Secret Third Question

Drive Strategic Action with a Cross-Functional Cadence

Fictional Examples of Driving Strategic Action:

Your metrics should flow from your unique business strategy

Example Non-NPS Questions

Takeaways

Change Region

North America

Europe

Asia Pacific