The value of open-ended questions is to get unaided and unbiased responses straight from the mind of the respondent. If I ask, “What is the first word or phrase that comes to mind when seeing or hearing Drive Research?” and you respond with, “Awesome service, passionate staff!” This tells me a lot of information in just a few words and you don't have to be a Research Analyst to know that. The depth of this response is far greater than if they were to select a radio button that reads as "Customer Service".
It can be a struggle to find themes or code open-ended responses, especially when you have hundreds or thousands of respondents. As a Research Analyst, I’ve got a few tricks up my sleeve when trying to find ways to add value to open-ended questions, especially the responses that are lengthy and vary from respondent-to-respondent.
Concerned about analyzing open-ended responses? Don't be! Read the tips and tricks below!
Here are 4 ways to add value to open-ended questions.
This is a tried and true way to add value to open-ended questions. To do this, I dump all of my open-ended responses into one column of an excel sheet. From here, I start assigning common themes, and it’s important to note that a response can have more than one theme assigned. Thinking about Voice of Customer research (VoC), these themes could be good customer service, quality product, easy to use ordering process, etc.
Once all responses have been coded, the researcher will typically present the findings in a summary chart that will feature the theme category, number of responses, and percentage. Alternatively, a graph can also be used.
This method can seem daunting if you have hundreds or thousands of responses. A researcher could spend a full day coding 3-5 open-ended questions with hundreds responses for each. For those of us working in market research, we've all been there. Depending on the scope of work, coding may be your best option, but there are a few other tricks that save time and provide compelling summative results to the clients.
#2: Word Count Text Analytics
This is a great way to add value to open-ended questions easily. For example, word counts are excellent for questions that ask for a word or phrase, specific trait, important improvement, etc. If this option is available to the researcher through their survey software, it’s worth every penny.
This method will automatically count repeated words, while excluding common words, like “the”, and present the findings sorted by most commonly used to least commonly used. Similar to coding, this lets the researcher quickly identify key takeaways without having to spend time reading and assigning a code to each response.
As discussed previously, this method is best for short responses. Using word count text analytics on long-winded responses will strip the meaning and value the original question was seeking an answer to. It doesn't allow for any context.
#3: Word Cloud
Similar to word count text analytics, this way of adding value to open-ended questions is great for short responses. Typically, word clouds are most effective when responses are only one or a few words at most. Similar to word count text analytics, most word cloud software has the ability to exclude common words.
As someone who appreciates graphics, I love this method. Users place all of the responses into the software (ensure common words are removed and words are not repeated) and then the software will make words that are more commonly used in the responses larger than less commonly used words. Some software will even give the user the ability to edit shape, font, color scheme, etc.
You can see the obvious issue with a word cloud when it comes to longer open-ended responses. Similar to word count text analytics, the true value and meaning behind these responses will get lost in translation.
#4: Theme Bullet Points
Unlike text analytics and word clouds, themed bullet points are ideal for long-typed responses that cannot be analyzed through a word cloud or word count text analytics. The researcher first reviews and cleans the responses and then incorporates them into the report. After the researcher reads through all responses, he or she can highlight key response themes in a bulleted format.
This method is useful when responses are somewhat similar and minimally differ. It's also the quickest way to analyze open-ends without taking additional steps. This method is usually accompanied with complete responses in the appendix of the report, which will provide the client with the opportunity to read each response individually.
Great analysis = Wow factor
What are the next steps?
After analyzing your open-ends you’ll likely need to incorporate them into the executive summary of your report. To do so, think about how presenting this information will be most effective. Similar to other means of summarizing results, consider the placement and how the level of detail can enhance or impede your results to the reader.
Contact Drive Research
Drive Research is a market research company serving Utica NY. We work with business and use data to fuel better decision making. Have questions about how we can assist your business or organization, or looking for more? All you have to do is ask.
Contact us at [email protected] or call us at 315-303-2040.