Practice makes perfect!
Survey writing is an art form. Several factors need to be taken into account when creating a survey. The top 6 factors market researchers keep top of mind when writing a survey are:
- Mutually exclusive close-ended answers
- Collectively exhaustive close-ended answers
- Bias wording and phrasing
- Order bais
- Survey flow
- Forgetting about the respondent
After years of practice, remembering these factors becomes second nature for market research pros when survey writing. If you are new to writing surveys, you may run into the 6 common issues with surveys written by non-researchers discussed below.
Keep calm. Survey writing tips from our market research company are below!
Issue #1: Close-ended answers need to be mutually exclusive
First, let's define close-ended answers for those who are new to market research survey writing terms. Close-ended answers occur in any type of question where respondents are given answer choices to select from.
Forgetting to be mutually exclusive when writing close-ended answer choices is a common issue with surveys written by non-researchers. This issue will lead to respondent confusion and errors with the collected data. Mutually exclusive answer choices is something to watch out for particularly with age, income, and other types of questions that are susceptible to overlapping answers.
Here's an example of a mutually exclusive issue.
Question: Which of the following best describes your age? Select one.
Answer choices:
a. Under 25
b. 25 to 35
c. 35 to 45
d. 45 to 55
e. 55 to 65
f. 65 or older
The issue here is that respondents who are aged 35, 45, 55, or 65 fall into more than one answer choice.
Here's how the answer choices should read:
a. Under 25
b. 25 to 34
c. 35 to 44
d. 45 to 54
e. 55 to 64
f. 65 or older
Now, all respondents will be able to select the appropriate answer choice no matter what their age is. *Sigh of relief*
Issue #2: Close-ended answers need to be collectively exhaustive
Forgetting to be collectively exhaustive with answer choices is another common issue with surveys written by non-researchers. This will lead to respondent frustration and gaps in the collected data. Simply put, collectively exhaustive means the answer choices cover every possible answer.
Here's an an example of a collectively exhaustive issue.
Question: Which of the following best describes your total annual household income?
Answer choices:
a. $25,000 to $49,999
b. $50,000 to $74,999
c. $75,000 to $99,999
d. $100,000 to $149,999
If respondents make less than $25,000 or more than $149,999 what answer do they select? Should respondents skip the question or pick the answer choice that's the closest to their actual total annual household income? Either way, there will be gaps in the data collected.
Here's how the answer choices should read.
a. Less than $25,000
b. $25,000 to $49,999
c. $50,000 to $74,999
d. $75,000 to $99,999
e. $100,000 to $149,999
f. $150,000 or more
The updated answer choices will ensure there are no gaps in the data collected and respondents will not be frustrated when completing the survey.
Issue #3: Bias wording and phrasing
Biased wording and phrasing in questions or answer choices can take a keen eye to watch out for. If there are issues with bias wording and phrasing in a survey there will be errors in the data collected and the results cannot be deemed reliable.
Here's an example of bias wording in a question.
Question: It's proven that Example Product is cheaper, better, and more sustainably sourced than competitors. Are you likely to recommend Example Product?
Clearly, this question is leading the respondent to give a positive answer.
Here's an example of bias wording in answer choices.
Question: Are you satisfied with the customer service provided by Example Company?
Answer choices:
a. Yes, I am extremely satisfied
b. I am somewhat satisfied
Clearly, again, there is bias. The answer choices are not balanced, meaning there should be negative answer choices in addition to the positive ones written above.
Want to learn more about bias? Here are 5 types of bias in market research.
Issue #4: Order bias
Order bias is a type of bias that occurs within the answer choices of a question. Respondents are more likely pick answers shown at the top of a list. To alleviate this, answer choices should be inversed/flipped or randomized to ensure order bias does not effect the data. A common example of this is inversing/flipping answer choices for rating or likert scale questions for each respondent.
In a previous post we covered everything you need to know about order bias.
Issue #5: Survey flow
Survey questions have an ideal flow. Typically, any type of screener question is asked first followed by the most important questions and ending with demographic questions. Keep important questions as close to the beginning of the survey as possible to increase the likelihood respondents will answer them if they decide to close out of the survey.
It's also important to ask more general questions before specific questions. For example, ask overall satisfaction before asking satisfaction with particular factors. This way, overall satisfaction is collected top-of-mind rather than after respondents have time to assess satisfaction of several factors which could effect their answer to the overall satisfaction question.
Issue #6: Forgetting about the respondent
Last but not least, do not forget about the respondent!
Surveys should take respondents less than 5 minutes to complete, which is typically 20 questions. When possible, try to make surveys more engaging with creative questions and using different forms of answer choices such as sliding scales, pictures/graphics, etc.
Contact Our Market Research Company
Drive Research is a market research company that serves clients across the country from Cleveland, OH to California. Our team helps organizations across the country with custom market research needs. Need help writing a survey or with the entire market research process?
Our team is here to help! Find contact information below!
Message us on our website
Email us at [email protected]
Call us at 888-725-DATA
Text us at 315-303-2040