There’s no such thing as a stupid question, right? Well, when it comes to market research methodologies such as online surveys, asking the silly questions can lead to a higher quality of data.
A red herring or trap question is an often-underutilized market research tactic in which a survey company includes odd questions mixed into a strand of normal questions. This allows data analysts to easily recognize survey respondents who fully read and remained engaged throughout the survey and those who only rushed to the end to collect their prize.
In this blog post, we’ll discuss four different examples of red herring questions to utilize in your next online survey.
Including red herring questions in a survey can lead to high quality research data. Use these four trap question examples in your next questionnaire.
1. Fake brand names
In any type of market research, it is important to find quality recruits who will offer the best insight into project objectives. Online surveys help filter potential participants by asking questions that will indicate who is the most qualified.
For example, Kellogg’s is looking to determine level of awareness for several of their cereal brands compared to competitors. Their target audience is those who buy boxes of cereal 3 to 4 times a month. A red herring question they could use would be:
The respondents who selected Marshmallow Brownie Crisps would be disqualified as this is not a legitimate brand of cereal and likely are not an avid cereal buyer. *SHAME*
Side note, if Kellogg’s is interested in creating a marshmallow brownie crisp flavored cereal – consider me their first customer. Also, consider doing some new product development market research.
2. Outlier answer to an opinionated question
Where an online survey company often recommends creating short and brief surveys online to avoid respondent burnout, sometimes there’s no avoiding long-form questionnaires. As a result, respondents may start clicking random answers to get to the end of the survey faster.
To spot these speedy survey takers, include an opinionated based question with an outlier answer included in the multiple-choice options.
For example, a non-for-profit organization is conducting a long online survey to better understand why people are and are not choosing to donate to their charity. An outlier red herring question to include that does not relate to the survey content may be:
Obviously, those who’ve selected basketball as their favorite ice cream flavor are not paying close (or any) attention to the questions of the survey. The answers of these respondents should not be included in the final data of the study.
3. Imaginary roles and responsibilities
However, knowing there is some type of incentive, respondents are more likely to exaggerate their expertise and responsibilities in an online survey to ensure they are chosen to participate.
For example, Cisco is renting a focus group facility to conduct a group discussion with data center managers and engineers of various organizations. A trap question to include in a pre-recruit online survey could be:
To any Joe Shmoe, this question is a head-scratcher, but for the type of focus group participants Cisco is looking for, this question is a no brainer.
Those who selected answer D, are not experts or knowledgeable enough in their role as this is not an actual brand of fiber enclosures, and should not move on to the group discussion. This is a great way to separate good from bad B2B participants.
Come on, Joe. Don’t be that guy.
4. Lack of basic knowledge to a common question
Survey research allows market researchers to collect data from a wide range of people. With a few clicks of a button, online surveys are known to be the most cost-effective and efficient way to hear from your target audience.
With that being said, the Internet is a crazy place and you can’t always be sure of who exactly is taking your survey. Like, robots. Yes, robots. One study estimates that more than half of all web traffic is now generated by automated bots.
More specifically, web bots are now able to take part in online surveys, usually for the purpose of receiving the provided incentive for completing surveys, without actually having to fill out the online questionnaire.
To ensure survey bots are not negatively affecting the accuracy of your survey research, consider including a question that should be a layup to human respondents, such as:
I know, math is hard – but not that hard. Including a red herring question as simple as this can validate the authenticity of survey respondents.
Hey survey bots, are you smarter than a fifth grader?
Drive Research is a market research company located in Syracuse, NY. Our team has the knowledge and tools to design a robust market research study, should it be the right fit for your business.
Interested in learning more about our market research services? Reach out through any of the four ways below.