There is no such thing as a stupid question, right? Well, when it comes to market research methodologies such as online surveys, asking the silly questions can lead to a higher quality of data.
A red herring or trap question is an often-underutilized market research tactic in which a survey company includes odd questions mixed into a strand of normal questions. This allows data analysts to easily recognize survey respondents who fully read and remained engaged throughout the survey and those who only rushed to the end to collect their prize.
In this blog post, our online survey company discusses four different examples of red herring questions to utilize in your next study.
Including red herring questions in a survey can lead to high quality research data. Use these four trap question examples in your next questionnaire.
1. Fake brand names
In any type of market research, it is important to find quality participants who will offer the best insight into the study's objectives. For instance, Drive Research utilizes screener surveys in our qualitative recruitment process.
With the help of online surveys, we are able to filter through potential participants by asking questions that will indicate who is the most qualified. This is a common tool in research studies such as focus groups, in-depth interviews, and mobile ethnography.
Oftentimes, our online survey company includes a red herring question in a recruitment screener to make sure people are honestly responding to the questions. For example, Kellogg’s is looking to determine level of awareness for several of their cereal brands compared to competitors.
Their target audience is those who buy boxes of cereal 3 to 4 times a month.
An example red herring question for this screener survey would be:
The respondents who selected Marshmallow Brownie Crisps would be disqualified as this is not a legitimate brand of cereal and likely are not an avid cereal buyer. *SHAME*
Side note, if Kellogg’s is interested in creating a marshmallow brownie crisp flavored cereal – consider me their first customer. But first, be sure to conduct new product development market research.
2. Outlier answer to an opinionated question
While our team often recommends creating short and brief surveys online to avoid respondent burnout, sometimes there’s no avoiding long-form questionnaires. This may cause respondents may start clicking random answers to get to the end of the survey faster.
As a result their responses become less legitimate. Because you are likely using survey data to make important decisions, it is critical the data be accurate. If your survey is longer that 15 questions, I highly encourage you to include a red herring.
To spot these speedy survey takers, include an opinionated based question with an outlier answer included in the multiple-choice options.
For example, a non-for-profit organization is conducting a long online survey to better understand why people are and are not choosing to donate to their charity.
An outlier red herring question to include that does not relate to the survey content may be:
Obviously, those who’ve selected basketball as their favorite ice cream flavor are not paying close (or any) attention to the questions of the survey.
The answers of these respondents should not be included in the final data of the study.
To learn more about creating engaging online surveys to reduce burnout, read this blog post.
3. Imaginary roles and responsibilities
A common way to find participants for more intimate and time-consuming market research methodologies like focus groups or in-depth interviews are to offer an incentive.
However, knowing there is some type of incentive, respondents are more likely to exaggerate their expertise and responsibilities in an online survey to ensure they are chosen to participate.
For example, Cisco is renting a focus group facility to conduct a group discussion with data center managers and engineers of various organizations.
A trap question to include in a pre-recruit online survey could be:
To any Joe Shmoe, this question is a head-scratcher, but for the type of focus group participants Cisco is looking for, this question is a no brainer.
Come on, Joe. Don’t be that guy. 🤦🤦
Those who selected answer D, are not experts or knowledgeable enough in their role as this is not an actual brand of fiber enclosures.
This is an easy way to disqualify respondents from a focus group discussion. Using red herring questions is a smart way to separate the good from the bad, especially when recruiting B2B participants.
4. Lack of basic knowledge to a common question
Survey research allows market researchers to collect data from a wide range of people. With a few clicks of a button, online surveys are known to be the most cost-effective and efficient way to gather feedback from your target audience.
With that being said, the Internet is a crazy place and you can’t always be sure of who exactly is taking your survey. Like, robots. Yes, robots. 🤖🤖
One study estimates that more than half of all web traffic is now generated by automated bots.
More specifically, web bots are now able to take part in online surveys, usually for the purpose of receiving the provided incentive for completing surveys, without actually having to fill out the online questionnaire.
To ensure survey bots are not negatively affecting the accuracy of your survey research, consider including a question that should be a layup to human respondents, such as:
I know, math is hard – but not that hard. Including a red herring question as simple as this can validate the authenticity of survey respondents.
Hey survey bots, are you smarter than a fifth-grader?
Watch This Video for More Red Herring Question Examples
In this video experts from our market research company, Drive Research will share what they think is the best red herring question to include in a survey and why.
Contact our Online Survey Company
Drive Research is a market research company located in Syracuse, NY. Our team has the knowledge and tools to design a robust market research study, should it be the right fit for your business.
Interested in learning more about our market research services? Reach out through any of the four ways below.
- Message us on our website
- Email us at [email protected]
- Call us at 888-725-DATA
- Text us at 315-303-2040
Emily Carroll
A SUNY Cortland graduate, Emily has taken her passion for social and content marketing to Drive Research as the Marketing Coordinator. She has earned certificates for both Google Analytics and Google AdWords.
Learn more about Emily, here.
