Quality is everything. I'll say it again, just in case you missed it and for extra emphasis... quality data is everything in market research. See the bolding there?
No business or organization wants to base important strategic decisions on bad data. Poor quality data comes in many forms. Those methodologies with more face-to-face and person-to-person fieldwork helps control data quality. However, it's hard to argue with the benefits of online research these days and with great power, comes great responsibility. In this case, quality checking data.
Low quality data can be a result of a variety of issues ranging from a poorly written survey to a low response rate to respondents who are not taking your survey seriously. Market research professionals have the tools to set projects up for success with effective survey methodologies and writing, but when you reach out to general consumers via online sources (e.g. social media, online panels, etc.) there are typically some bad responses that sneak through.
The trick now becomes how to identify a bad response from a good response so it does not infect your high quality data. Data quarantine, here we come.
Don't worry - we'll give your the tools to find your bad data!
Here are 4 types of market research respondents to keep an eye out for when you are quality checking the data in your next online survey. Each type of respondent creates it's own unique set of problems for a market research firm.
The Speeder
Always keep an eye out for Herman, one of the 4 online survey respondents that scare market research firms!
Meet Herman, the classic survey speeder
Herman is eager to participate in research, so much so that he flies through surveys without taking the time to fully read questions answers to ensure he's answering questions correctly.
Tips to Handle the Speeder
An easy way to see if Herman has responded to your survey and infected your data is to analyze response times. Researchers typically find the time it took respondents to complete a survey when exporting paradata, which is found within the raw survey data file.
Wondering how long your survey should take to complete? Ask a colleague, or a few colleagues, to complete the survey and see how long it takes it complete. If it took your colleagues on average 3 minutes to complete the survey and a respondent finishes it in 45 seconds, it's most likely Herman sneaking into your data. Another blanket technique is to shave off the outliers (the top and bottom 1% or 2% of survey response times).
Another way to check for Herman is to see if there is any flatlining survey respondents. Flatlining is where respondents answer multiple ranking questions with the same response. An easy way to check for flatlining is to review the responses to these types of questions.
Lastly, some speeders will leave random responses in open-ended questions. To check for this, simply review all responses to open-ended questions. If it doesn't make sense or is just randomness, it's most likely Herman.
The Professional
You may want to see if Jackie responded to your survey.
Enter Jackie: The professional market research participant
Jackie loves to participate in market research and is a member of various online research panels. She likes to earn rewards for participating in research, but also answers surveys just for fun and for a chance to win a sweepstakes. She loves to share!
Since Jackie participates in market research so often, this can make her responses prone to bias, which is a dreaded word in market research. While professional survey takers aren't necessarily bad for every study, it's important to keep them in mind and consider how it can affect your data (which is why she is one of 4 online survey respondents that we watch out for).
Tips to Handle the Professional
An easy way to see if Jackie responded to your survey is to ask! Add, "Have you participated in a market research study in the past 6 months?" Or inquire about studies that are directly related to your topic (either methodology or industry). If the answer is yes, Jackie will be screened out.
The One Who Doesn't Read Directions
Find out if Chad responded to your survey!
Chad doesn't always follow directions
Chad may read through some of your questions and answers fully, but also tends to skim questions and not follow directions. This leads to, you guessed it, bad data, making him one of the 4 online survey respondents that scare market research firms.
Tips to Handle the Respondent Who Doesn't Follow Directions
One way to find out if Chad responded to your survey is to add a quality check question in your survey. Often times, these quality checks are within a matrix of ranking questions. These are commonly called red herrings in the market research industry.
For example, a question will ask you to rank several factors on a 5-point likert scale, and within those factors one says, "Please select choice 4." Anyone, who doesn't select choice 4 should be removed from your data set...
Gotcha Chad!
The Tech Wiz
Is Olivia lurking in your data?
Check to see if Olivia is in your data!
Once you complete a survey, you don't complete it again right? Oftentimes, survey software has the ability to disable a respondent to click a survey link more than once to prevent them from submitting more than one response. This done through cookies.
Respondents may try to submit more than one response to a survey for a variety of reasons such as entering a survey incentive more than once or attempting to qualify for a survey.
Duplicate respondents are scary, which is why Olivia is one of 4 online survey respondents that market researchers watch out for.
Tips to Handle the Tech Wiz
To check to see if Olivia is in your data, see if there are any duplicate IP addresses. This data is typically exported when researchers the full data set, which includes paradata.
It's important to note that it's possible for two different people to complete a survey from the same IP address. To avoid deleting a valid response, review the responses for similarities. If the response is too similar, it could be a duplicate. You may see duplicate IP addresses from those who work at the same company and complete a survey.
Another way to check for duplicates is to review any contact information provided by responses. If you collect respondents names, email address, or phone number, check for duplicates in this data for removal.
Contact Our Market Research Firm
Our team at our market research firm manages a number of online survey projects in New York State and from coast to coast. Have questions about an upcoming online survey project or need a quote from our online survey firm?
Contact us at [email protected] or call us at 315-303-2040.