In 1853, U.S. Commodore Matthew Perry (no, not the Friends actor) sailed to the shores of Japan to strongly suggest (with several gun-laden vessels) that the ruling shogunate open Japan’s ports to outside trade. For 200 years, Japan had embraced a policy of near total isolation from the West, but with the Industrial Revolution fresh out of the oven, even isolationist Japan couldn’t ignore the benefits of trade. What does this history lesson have to do with Product Managers and Customer Success?
Morgan is 15-year startup marketing vet, who is passionate about helping great ideas and companies grow. He’s currently the COO of Inman, a real estate media company that covers the residential real estate space in the U.S. He’s also married to an awesome woman and has two kids that remind him of how lucky he is every day.
He loves to study what makes online companies grow, what makes some brands successful and others not, consumer behavior and all things business. He reads a ton, with his favorite reads being business books and biographies of famous leaders. His latest read, Leadership BS, is a great book on how most of the leadership advice is misguided and harmful to leaders and their teams.
You can follow Morgan on Twitter at @morganb.
- How have you used qualitative research in your career as a growth marketer?
Qualitative research for me really lies at the heart of what it means to be a marketer.
Talking to customers, listening to people, peeling back the onion with the right questions to really assess their needs, wants and motivations is incredibly powerful. Running growth at Qualaroo certainly meant a lot of time studying and using qualitative research. It really helped me see first hand how powerful it can be when deployed properly.
I’ve used it in my career in all sorts of ways: conversion rate optimization, product development, marketing campaigns, website and email copy, and software package and pricing. It’s helped me learn what matters most to people, what concerns people have, and the words they use when they talk about a product and solution.
- What are the differences between qualitative and quantitative research?
Analytics tells you what’s happening, but voice of customer research tells you why. It’s that contextual layer that helps put data into perspective. You can stare at numbers and guess at what’s happening, or you can ask your customers and get feedback about what’s really going on.
- What is the importance of qualitative research for online businesses?
The beauty of qualitative feedback is that it helps you define the problem space better when you’re looking to improve the performance of your business. For example, you can see a really high cart abandonment rate and guess at all the potential issues preventing someone from purchasing or subscribing, or you can talk to some people and narrow your set of potential hypotheses and experiments. It’s a great way to shorten your experimentation process and find higher quality tests that are more likely to be wins.
- What methodologies can be used to conduct qualitative research? How can a business make sense of all the feedback they get ? We think some people inherently distrust qualitative, don’t know what to do when they get A LOT of it.
There are lots of ways you can collect qualitative feedback, from interviewing people in person or on the phone, surveying users via an onsite survey, using Net Promoter Surveys via companies, to marketing panels where you survey people who are in your target market. The key to effective user surveys is to ask the right questions to the right people, so that you’re eliciting valuable feedback.
If you use a Net Promoter Score survey, you’ll get a numeric representation of your overall customer happiness or satisfaction with your service. You can benchmark yourself over time and also to other companies who publish their NPS online. It gives you some context to the results you’re getting and helps you gain some understanding of the feedback you’re getting.
You don’t necessarily need a ton of feedback from qualitative research to find answers that can help your business. Jakob Nielsen, one of the foremost user experience experts, wrote an article 15 years ago that showed that getting feedback from just five users uncovered 85% of usability problems. So you don’t need a ton of feedback to uncover important learning that can improve your growth, conversion rates, etc.
- What are some of the common mistakes people make (or fears? resistance?) when conducting qualitative research and how can they be avoided?
The two biggest mistakes are: not doing qualitative research in the first place and then not putting it to use. People are afraid that they’re going to ask the wrong questions, get a non-representative sample, hurt their conversion rate and more. The third big mistake is asking the wrong people for feedback.
These are all easily avoided. The first is to just do it. Qualitative feedback doesn’t have to be hard. Just start by talking to customers, on the phone, via email, onsite, in surveys, etc. Getting over the fear of doing it is the hard part. Then, once you have data flowing in, it’s important to share it with your team.
Finally, asking the wrong people is a big issue. You want to ask people who are potential customers, not just random people who never have any intent or landed on your site by accident. Just like you wouldn’t do customer development interviews with someone outside of your target audience, don’t waste your time getting feedback from people who will never buy.
To do this, filter out unqualified traffic, users, etc. For example, at Inman we don’t run onsite surveys to older content which gets great SEO traffic but is written for a different audience. Those readers are not the people we’re trying to build our business with.
- How can qualitative research be used to find language-market fit, problem-solution fit, and product-market fit?
This is where qualitative research really shines. For language-market fit it gives you insight into the exact words and positioning they use to describe your product. You can use their words to help with copy and positioning that is relatable and intuitive to your target audience.
For problem-solution and product-market fit you can ask questions like ‘What product would you use if you didn’t use ours?’ or ‘How disappointed would you be if our product was no longer available?’ or ‘What’s the one thing that would make this product indispensable?’ and more to gauge how important your product is to your users and how to make it a must-have.
One of the best ways to improve retention is to ask people who leave why they left and then work backwards to solve those issues and keep people engaged and happy with your product.
- How can it be used to help scale a startup once product-market fit has been established?
Qualitative feedback is essential to conversion rate optimization. If you have PMF, then you want to maximize your growth through improved conversion rates, messaging, acquisition, retention and more. Qualitative research can give you the insights you need to improve all of those things.
One of the best qualitative questions to ask as part of conversion optimization is to ask people who just successfully purchased or signed up this question, ‘What almost stopped you from signing up/purchasing?’
The answers you get from your new customers will point you directly to the moments of confusion and hesitation that almost derailed them. If you go back to those spots in the funnel, you’ll likely find many more people who couldn’t get over that hurdle. That intelligence gives you great insight on where to start experimenting to improve your overall conversion rate.
Start gathering qualitative feedback today. Signup for free in-app Net Promoter Score with InMoment.
- 2x response rate versus email provides a more complete picture of the Entelo customer base
- Real-time feedback helps Entelo jump on customer issues quickly, rather than letting problems go unaddressed for months at a time
Editor’s Note: This blog was originally posted on CX Cafe’.
Your respondents might know more than you think.
Including a “don’t know” option in a survey is an issue that is currently under speculation. The “don’t know” option can be explicit, as shown with the scale, or it can be implicit by the use of skip patterns within a survey. It’s a powerful option to give survey takers who don’t really know the answer–an option so they don’t get frustrated, but it also can serve as a cop-out for those who just don’t want to answer the question. So where do you draw the line?
The “don’t know” option can contribute to good survey design, because it utilizes skip patterns to alleviate the need of showing respondents a set of questions that are not applicable. However, if the “don’t know” option is associated with attitudes concerning relevant touch points or facts, you may want to reconsider including that option in your surveys.
So what happens if you include the “don’t know” option in your Survey?
- First, when that option is present, respondents are more likely to select it than engaging in the question.
- Second, researchers have found that respondents do a pretty good job at answering questions in the face of uncertainty. For example, if a fact-based question had four choices, respondents who initially said don’t know had much higher accuracy than the 25% that guessed at random.
- Third, attitudes can be more reliably “guesstimated” than facts.
- Fourth, if respondents choose “don’t know,” multivariate analysis requires those answers to be treated as missing, so the data is not inaccurate. For missing values, we often use methods to try to recover those answers (imputation). Who do you want to estimate those underlying values? The researcher? The respondent?
- Fifth, and finally, placing a “don’t know” option on a crowded scale or not setting it apart from equidistant scale points can lead to respondent confusion and incorrect selections.
The fear of not using “don’t know” is that you are forcing the respondent to provide meaningless responses. However, the use of “don’t know” can lead to MORE data problems. In general, minimize the use of “don’t knows” in your surveys, for a more powerful and informative survey.
Note: There are some considerations about omitting the “don’t know” option on mandatory questions. If there are too many questions which force the respondent to answer, the respondent could get more frustrated without the “don’t know” option. Depending upon the questions you are asking on your survey, it is key to find a healthy balance between adding “don’t know” on your survey and taking it off.
In this age of survey fatigue, getting users to engage with a survey in any medium is challenging. Mobile apps are no exception, and have their own unique constraints. The good news is that in-app surveys can provide a streamlined mobile experience that results in super-high response rates and meaningful feedback, too.
Asking for survey response on a mobile screen can create a friction-y experience for users.
Low screen attention. Small, cramped mobile screen. Tiny text. Question after question. Who wants to deal with that?
You need a streamlined survey solution that reduces friction but still provides rich feedback. So, how do you overcome the constraints of mobile?
Net Promoter Score surveys minimize friction.
Net Promoter Score (NPS) is recognized as a powerful measure of customer happiness, and a lean, agile way to elicit meaningful feedback from users. Not familiar with NPS? Here are the basics.
The NPS survey consists of a single survey question, plus an opportunity for a quick qualitative response. Because of its simplicity, the NPS survey really shines in the mobile context. Have a look at how it works here on Android and iOS.
Surveying mobile users via email can mean low response rates.
Until now, mobile businesses have had to rely on email surveys to get NPS user feedback, and email certainly has its place. Trouble is, app developers may not have a user’s email. Even when you do, inboxes are noisy places and readers are less likely to click through from mobile devices. Also, an NPS survey via email arrives after the fact, after your user has left your app. Their attention is elsewhere.
For high response rates, ask the powerful NPS question right in your mobile app.
You can now show your user an NPS survey in real-time, when he or she is engaged with your app on a mobile device. Users are scoring and commenting in context – feedback is fresh and relevant, which helps make it more actionable. Typically, you will see a 40-60% response rate right off the bat.
Surveys can be triggered to suit your business needs. For example, a user could see a survey after she has logged in x number of times, taken a specific action, or 30 days after downloading the app.
Using an external platform to manage your NPS process has its advantages. It can scale easily, is hassle free to try and deploy, and frees up resources to focus on the “so what?” — leveraging NPS results to improve your application. A streamlined version of an NPS platform can come free and shouldn’t break your budget. Our tool, Wootric, is one example.
Get the ebook, The Modern Guide to Winning Customers with Net Promoter Score. Learn eight ways to leverage Net Promoter Score for customer loyalty and growth.
To maintain strong response rates, take action on user feedback.
Maybe you filled out a survey once — really took the time to give constructive feedback. Did you hear back from the company? If you did not, how likely are you to fill out another survey from that company? Not very likely, right?
For you to continue to garner high survey response rates, your mobile users must know that their feedback matters. An NPS platform dashboard makes it easy for you to respond. The dashboard is where you can monitor the cumulative Net Promoter Score your app is earning, and slice and dice your data. It is also where you can see individual scores and qualitative feedback from individuals.
If your users have accounts and you have the resources, you should respond directly to individuals. You can do so right from your NPS dashboard. You can also forward feedback to a team member in Customer Support, Customer Success, or Product Management for further action. To streamline the process, you might automate an email response to the bulk of respondents depending on whether they are Promoters, Passives or Detractors.
If your users are anonymous, at a minimum, you can acknowledge in software release notes that it was user feedback that revealed that recently-squashed bug, or drove the development of xyz feature.
High response rates and rich user feedback are possible in the mobile environment.
The streamlined nature of the NPS survey is a great fit for the low-attention span of the mobile user and the constraints of the small screen. Consider in-app NPS surveys for higher response rates than email. Be sure to show your users that you listen to their feedback, they will be more likely to answer another survey down the line.
Start measuring Net Promoter Score in your mobile app for free with InMoment
When it comes to using Net Promoter Score surveys to gain insights from your customers, you probably have questions about sampling.
How do you decide how many customers to survey? When should you first survey a customer? What about after that? These are three important questions to think about in advance of getting started.
In this post, I’ll discuss best practices for survey sampling for NPS. While these practices apply to many types of businesses, I’ll relate them to gathering customer feedback in the online world — inside web and mobile applications, and on websites.
Setting up an NPS program? Download the free ebook, The Modern Guide to Winning Customers with Net Promoter Score.
As a startup founder, you were probably on a first name basis with many of your early users. Some became cheerleaders and champions, others churned. And, in those early weeks and months, everyone’s detailed, anecdotal feedback was critical. Those first 5, 10, or 50 customers helped you hone your product into something that hundreds or even thousands of customers now rely on.
As more and more users come on board, you can no longer develop personal relationships with each and every one. Plus, your attention is pulled to hiring, infrastructure and funding. Now, more than ever though, you need to know what customers think of your product and stay connected. You need a solution that scales.
What do you do when you can’t talk to every customer any more?
Homebase, a startup that offers free employee scheduling and timekeeping software, has thousands of customers, uses Net Promoter Score to continue its personal response to customer feedback and to shape the product roadmap as the company grows.
The beauty of Net Promoter Score is its simplicity. At the core, it is one key numeric ranking question supported by an open ended “why” question. (See my post on the ABCs of NPS to learn the basics.) It’s a question that can be asked in many channels: over the phone, in an email, in your website or mobile app, in person. A variety of tools exist to help you execute NPS for your business with various degrees of complexity and cost. These range from setting up your own email survey to using an automated service to hiring a consulting firm to implement NPS programs for your company. Here’s a snapshot of what these different options have to offer:
Whether you specialize in customer experience, engagement, success, or service, you’re tasked with retaining and delighting customers all the time. Plus, you have to get to know them.
That’s why we talked to 6 customer engagement experts to find out what strategies bring the most success. Here are their top tips:
1. Using Tools as You Scale
At Grasshopper, we struggled with finding ways to engage our customers as we grew. It became a real challenge for us to develop and maintain strong connections with our customers the way we used to: sending welcome packages, notes and swag to people we spotted on social doing cool things or giving us shout outs.
We realized that what we were doing was becoming harder and harder to scale, Read More…
Imagine a lion tamer in the center of a circus ring, whip in one hand, a wooden chair in the other. The lion stares at the man, unsure of its next move. The crowd waits anxiously for the lion to strike — but nothing happens. The angry lion has been tamed, without brute force or coercion.
The secret, as writer James Clear details on his blog, to taming a lion isn’t submission. Rather, it’s confusion.
When a lion tamer holds a chair in front of the lion’s face, the lion tries to focus on all four legs of the chair at the same time. With its focus divided, the lion becomes confused and is unsure about what to do next. When faced with so many options, the lion chooses to freeze and wait instead of attacking the man holding the chair.
This is precisely the reason that Net Promoter Score is such an effective tool. In other words, the goal isn’t to “tame” your customers but rather to set them free. Read More…
Update 11.30.2015: MOBILE Now you can use Segment.com to enable Wootric’s Net Promoter Score platform in your mobile app without touching code or waiting for the next release cycle. Learn more about Segment’s mobile integration platform.
In-product tools can be extremely valuable to a business. Google Analytics, Mixpanel, Optimizely, Crazy Egg…I’m sure we all have a few we can’t live without. But if your technical resources are constrained, even the easiest tools to install end up somewhere in the development pipeline. As the non-technical side of the house, I know how hard it can be to wait to test out a new shiny tool.
We built InMoment to measure Net Promoter Score inside your product. Why? Because every online business deserves to hear from more customers in the most fresh and contextual way. While a developer would tell you Wootric is a fast code install, it’s still an install. And I’d certainly tell you that “marketing me” would need help.
We succeed when you succeed, so we’ll remove any barriers we can to help our customers–technical or non-technical–quickly get up, running, and listening to customers. Read More…