Understanding how many people to survey is a critical step in market research. Is 50 responses enough? 100? 250? 1,000? Regardless of how many people you survey, one thing is certain: The more people you survey among a random population, the more reliable the data becomes.
But what is the sweet spot? When I say sweet spot, I mean the ultimate number or best-practice number which brings several variables together in harmony like budget, timeline, reliability, analysis, etc.
400 responses is considered the sweet spot for the number of responses in the market research industry. Why? This is driven by several factors but the 3 most important are the margin of error, cross-tabulations, and the law of diminishing returns.
Our market research company explains in more detail below.
Here are the reasons 400 survey responses is the magic number in market research.
Learn more about the 3 reasons which drive the recommendation of 400 survey responses for your market research questionnaire. If you have enough sample to invite, the goal number of responses should be 400 as it is considered best-practice in the industry.
As stated this highly depends on the sample you have available. If you have 500 customer email contacts, it is unreasonable to think you will obtain 400 responses. However, if you have 100,000 customer emails, you may want to think about pulling a random sample of customers to reach 400 and only 400 responses for your survey. This saves you from reaching out to too many customers all at once.
Use this handy margin of error calculator to understand margin of error among different sample sizes for your survey.
Margin of Error
Obtaining 400 responses to your quantitative survey offers a margin of error of +/- 5% at the 95% confidence level. What this means is if the survey were conducted with a random population 100 times, 95 out of the 100 times the results would yield within +5% or -5% of the stated totals.
Awareness of your brand is 80%? If you conducted the survey again it would range between 75% and 85%. Satisfaction with your customer service is 45%? If you conducted again, it would range between 40% and 50%. A 5% margin of error offers a high degree of statistical reliability for your market research.
One of the largest advantages to having a big sample size is the ability to run cross-tabs. This is another driver to collecting 400 responses. If your sample size is 100 in total and you want to compare data between males and females you may be looking at base sizes of 50 or less for each. The statistical reliability of each declines as well.
Or let's say you want to compare feedback from our survey across age groups. If you have multiple ranges of age groups (18 to 34, 35 to 54, and 55+) you are splitting up your total number of responses even further. Larger sample sizes increase opportunities and reliability in cuts and cross-tabs for analysis.
Cost Trade Off and Diminishing Returns
If you want more responses it will cost you more. Whether this is more time to send reminders, more sample to purchase, or more responses to pay for, all equal additional costs. Shifting from 100 responses to 400 responses will decrease your margin of error from +/- 10% all the way down to +/- 5%. So one could easily argue this jump in sample size and reliability is worth the cost.
However, a jump from 400 responses to 500 responses only decreases the margin of error by 0.5%. So one could argue the increased reliability of 0.5% is not worth the additional cost of 100 responses. Therefore many experts in the industry consider anything beyond 400 survey responses in total is a bit of diminishing returns or ROI.
Also, keep in mind cross-tabs from our previous point. If you are making major marketing and strategic decisions for specific age groups or regions of the U.S. you may want to consider larger sample sizes for each sub-segment.
For example, if you run separate marketing campaigns for different regions of the U.S. (West, Midwest, Southeast, Northeast) you may want to consider obtaining 400 responses for each region to reach the +/- 5% margin of error. This would result in a total sample size of 1,600.
Contact Our Market Research Company
Thank you for visiting our market research company blog. Our market research company works with a variety of industries and companies across the U.S. Our market research services include online surveys, focus groups, and intercept surveys, among others.
Need a proposal for your survey? Contact us 1 of 3 ways below.