With the emergence of online and mobile surveys particularly through market research panels, satisficing has become a common concern for market researchers.
Respondents are time-poor and should they choose to participate in surveys, they expect them to be fast and easy to complete. However, there is such a thing as surveys being completed too fast, which in turn leads to poor data quality.
In this blog post, our online survey company discusses what satisficing is, examples to look for, and how to prevent it.
What is Survey Satisficing?
Survey satisficing is when the survey respondent takes short-cuts to complete a survey faster. This may include knowingly clicking incorrect or random answers in order to finish quickly.
A study conducted by Jon Krosnick and David Vannette at Stanford University identified two key drivers of satisficing in surveys:
- Lack of motivation from respondent
- The difficulty of the task
In short, survey satisficing occurs the most when the respondent had little motivation and the task was deemed difficult.
Example of Survey Satisficing
The online survey respondent Tom is asked: "Have you been to McDonald's in the past 3 months?" He is given the option to click yes or no. Tom visited McDonald’s last night, so he immediately clicks "Yes".
Tom is then asked a series of questions about…
- Satisfaction
- Purchases
- Perception of the restaurant
- Likelihood to go again
This series of questions takes him 5 minutes to complete.
Once finished with that series he is asked: "Have you been to Burger King in the past 3 months?"
Even though Tom went to Burger King last week, he opts to click "No." He wants to avoid the additional 5 minute series of follow-up questions to complete the survey faster and earn his reward.
How to Prevent Satisficing in Surveys
Market research professionals are trained to prepare for these survey satisficers and develop standards into each survey to counteract these data fiends.
For example, Drive Research follows a strict data quality checklist to help identify poor survey responses and remove them from your data file before analysis.
Here are 5 ways we prevent respondents like Tom from ruining your data quality:
Option 1: Drop a Red Herring in Your Survey
A red herring question is a quality control measure in surveys where unconventional questions are inserted among regular ones to identify respondents who are thoroughly reading and engaging with the survey versus those who are not.
This may be as blunt as asking the respondent to click answer "C" in a series of grid questions. Adding this in your online survey helps identify speedsters.
Watch our video for more red herring example questions.
Option 2: Using Extreme Opposites
This option we don't usually like at Drive Research because it can involve double negatives. However, many survey writers and designers use it.
This would involve asking a respondent to rate the level of agreement from strongly agree to strongly disagree on the following: "The Big Mac is the most tasty menu item at McDonald's."
To monitor data quality the survey would then ask the opposite later in the series: "The Big Mac is the least tasty menu item at McDonald's."
If the respondent rated both strongly agree or both strongly disagree it will raise a red flag in your analysis because both options cannot be true.
Option 3: Check for Straightlining
Survey takers hate long series of 1 to 10 rating scales. It’s why we often say, ‘Repetition is the killer of survey engagement.’
Once the survey participant reaches his or her threshold, they may begin just clicking "9" for every question. Keep an eye out for these straightliners and remove them from your data set.
Option 4: Review Time to Complete the Survey
You've most likely tested your survey thoroughly and as the survey designer, you know what range of time it will take someone to complete it.
During fieldwork you can even calculate your average time to complete. If the survey takes an average of 12 minutes and you have a handful of respondents that have completed it in 3 to 5 minutes, take a closer look.
Option 5: Limit the Number of Selections
In an online survey, it's crucial to provide a comprehensive list of answer choices, but it's important to do so thoughtfully to maintain respondent engagement, particularly for mobile users.
For example, when asking respondents to identify fast food restaurants they are familiar with, it's more effective to limit the list to the top 6 to 8 most popular options and include an "other(s)" category.
This approach saves time and keeps the survey manageable.
Whereas, providing an excessively long list, such as 78 fast food restaurants, can overwhelm respondents, leading to satisficing and higher dropout rates, both of which can compromise the quality of your data.
Contact Our Online Survey Company
Drive Research is a global market research company specializing in online surveys. Our team of expert market researchers follow strict data quality guidelines to spot poor survey respondents before their feedback is included in your research analysis.
To learn more about our online survey services, contact Drive Research today.
- Message us on our website
- Email us at[email protected]
- Call us at888-725-DATA
- Text us at 315-303-2040
George Kuhn
George is the Owner & President of Drive Research. He has consulted for hundreds of regional, national, and global organizations over the past 15 years. He is a CX-certified VoC professional with a focus on innovation and new product management.
Learn more about George, here.
