Editor’s Note: This blog was originally posted on CX Cafe’.
Your respondents might know more than you think.
Including a “don’t know” option in a survey is an issue that is currently under speculation. The “don’t know” option can be explicit, as shown with the scale, or it can be implicit by the use of skip patterns within a survey. It’s a powerful option to give survey takers who don’t really know the answer–an option so they don’t get frustrated, but it also can serve as a cop-out for those who just don’t want to answer the question. So where do you draw the line?
The “don’t know” option can contribute to good survey design, because it utilizes skip patterns to alleviate the need of showing respondents a set of questions that are not applicable. However, if the “don’t know” option is associated with attitudes concerning relevant touch points or facts, you may want to reconsider including that option in your surveys.
So what happens if you include the “don’t know” option in your Survey?
- First, when that option is present, respondents are more likely to select it than engaging in the question.
- Second, researchers have found that respondents do a pretty good job at answering questions in the face of uncertainty. For example, if a fact-based question had four choices, respondents who initially said don’t know had much higher accuracy than the 25% that guessed at random.
- Third, attitudes can be more reliably “guesstimated” than facts.
- Fourth, if respondents choose “don’t know,” multivariate analysis requires those answers to be treated as missing, so the data is not inaccurate. For missing values, we often use methods to try to recover those answers (imputation). Who do you want to estimate those underlying values? The researcher? The respondent?
- Fifth, and finally, placing a “don’t know” option on a crowded scale or not setting it apart from equidistant scale points can lead to respondent confusion and incorrect selections.
The fear of not using “don’t know” is that you are forcing the respondent to provide meaningless responses. However, the use of “don’t know” can lead to MORE data problems. In general, minimize the use of “don’t knows” in your surveys, for a more powerful and informative survey.
Note: There are some considerations about omitting the “don’t know” option on mandatory questions. If there are too many questions which force the respondent to answer, the respondent could get more frustrated without the “don’t know” option. Depending upon the questions you are asking on your survey, it is key to find a healthy balance between adding “don’t know” on your survey and taking it off.
