While you build your survey you have two options for survey design: (1) open-ended questions or (2) close-ended questions. An open-ended question is a free text box which allows a survey respondent to type in a response while a closed-ended question is when a respondent is forced to choose among shown answer options. Both have their own pros and cons for both the market research firm and the survey respondent.
The good news is you only have 2 options. The bad news is you only have 2 options.
In this post I'll cover the pros and cons of open-ended and closed-ended questions for both the survey writer and the survey respondent. When you review pros and cons you need to look at both audiences. At the end of the post it will be up to you to determine the correct sequence of questions which work best for your survey including the number of open-ended and closed-ended questions.
As a side note first ask yourself if you want to explore or measure in your market research study. Exploring lends more towards open-ended questions and qualitative market research. Measuring lends more towards quantitative market research and close-ended questions.
Pros of Using Open-Ended Questions for the Survey Writer
Here is a list of pros for open-ended questions for the survey design team.
Unsure of the exact answer categories to list for your question? This is one of the benefits of free text boxes. Instead of biasing the respondent through the answer categories offered, a free text box will present you with unbiased feedback.
Everyone loves data visualization and graphical displays of data. Collecting open-ended responses provides the survey writer with the flexibility to create a word or image cloud based on responses. The words used the most are detailed in larger fonts. Want to learn more about word clouds in market research? Go here.
Similar to word clouds, open-ended question provide the analyst with additional flexibility for reporting. Word counts tally the number of times a specific word was typed in the open-ended comment box (e.g., did you know 67% of customers mentioned the sales rep Dan S. when asked about what they liked best about buying from Company Inc.)
Yet even more flexibility in analysis, sentiment addresses the feelings or emotions behind a typed response. Is the open-ended comment positive or negative? Sentiment analysis helps measure this for the analyst rather that relying on radio button selections with no context.
The MECE principle in market research teaches us being exhaustive is an important part of survey design. By using an open-ended comment box it ensures the question responses will be exhaustive. A con of using closed-ended questions is running the risk you forget to list an important category for selection. If you head down this close-ended path, always include an "other" if warranted.
Cons of Using Open-Ended Questions for the Survey Writer
Here is a list of cons for open-ended questions for the survey design team.
Every analyst's favorite task right? Probably not. When you ask open-ended questions, you'll be forced to read through all of the responses and categorize them into themes. These themes are then built into charts and graphs. It requires a lot more work than running a frequency chart on a closed-ended question.
Letting the respondent type out a comment will undoubtedly lead to misspellings and grammatical errors. To clean up the data file and report, you'll want to go through and clean these open-ended comments from your respondents. An unfortunate "must-do".
Let's face it, sometimes survey respondents are lazy. Too many open-ended comment boxes and you'll definitely see some drop-off in your survey. Survey respondents want it easy and in many cases "easy" means select from a choice of radio buttons.
Design your online survey with the user and respondent in-mind. This translates to mobile experiences.
Pros of Using Closed-Ended Questions for the Survey Respondent
Here is a list of pros for closed-ended questions for the survey respondent.
Oftentimes respondents like free text boxes because it's free reign for them to type whatever they'd like. They are not constricted by the categories you've shown previously and they now have a chance to voice their opinion straight from their heart. Many times, they like this and if it's not offered before the end of the survey, they often feel like they could not share the whole story.
Respondents can only take so many radio buttons, boxes, circles, slider scales, or grid ratings. Placing an open-ended question in a survey which requires them to think about what they want to write helps promote engagement and interest if not overused.
Cons of Using Closed-Ended Questions for the Survey Respondent
Here is a list of cons for closed-ended questions for the survey respondent.
Open-ended comments take type to type. So several of these added to an online survey will add significant time. This adds to the length of interview (LOI). Now a days, respondents don't have more than 3 to 5 minutes to share feedback. Surveys need to be quick and to the point. Open-ended questions don't always help with this.
Typing open-ended comments is a lot easier with a keyboard, more difficult with a tablet, and much more difficult with a smartphone. Considering half of all online surveys are completed via mobile device, open-ended questions are a large concern because it can become frustrating to type and continually have to go back and correct (for those who care enough to). For those who don't see Con #2 above for the survey writer.
As a result of this mini-frustration, open-ended questions often drive drop-off on online surveys.
Strike the right balance of open-ended and close-ended questions in your survey. The ratio we use internally is about 1 open-ended question for every 10 to 15 questions. Any more than that and the survey is not designed with the mobile user in-mind. Open-ended comments provide all kinds of flexibility in analysis and reporting but you must first secure survey respondents and get them to complete your survey before you worry about post-fieldwork options.
Contact Drive Research
Drive Research is a Voice of Customer (VoC) market research company located in Upstate NY. For more information about our mail, phone, and online surveys we use for VoC, reach out to us at 315-303-2040 or email us at firstname.lastname@example.org.