Tips: How many people should we survey

How many people need to take your survey to make it statistically significant?

Written by Mikhail Dubov
Updated over 3 weeks ago

In a perfect world, you’d want to hear from your entire user base. Unfortunately, this isn’t possible—not all users respond to surveys. The next best thing is to ensure that you hear from a statistically significant portion of your user base. This topic lists some tips to determine the right sample size for your survey.

• Determine a Sample size
• Adjust the Sample size
• Minimize Sample size errors

Tip #1: Determine a Sample size for your survey

Sample size is the total number of completed responses a survey receives. The Sample size for a survey is determined based on the following factors:

• Population size: Total number of people whose opinions you want to know through the survey
• Margin of error: Percentage of error you expect in the survey results
• Confidence level: Percentage of confidence that the survey results will fall within the margin of error
• Standard Deviation: Estimated percentage of variation anticipated in the survey results

You can determine the Sample size for a survey by hand using an empirical formula or by using Chattermill’s sample size calculator. See the Sample size column in the following table, as an example.

After you determine a Sample size for the survey, reach out to as many users as required to receive responses that meet or exceed the Sample size.

Tip #2: Adjust the Sample size based on the use case

Surveys can be conducted for a multitude of purposes across various industries. For Customer Satisfaction surveys, you can act on the insights even if you haven’t received responses that meet the sample size. Unfortunately, this doesn’t apply to surveys related to employee satisfaction or market research. Consider waiting for responses that meet or exceed the sample size before acting upon the insights. Without a statistically significant number of responses, you may not be getting a holistic view of the findings and insights.

Tip #3: Minimize Sample size errors

Sample size errors can negatively affect the credibility of survey results. They can be minimized by reaching out to users who are representative of the user base you wish to survey. See our blog for some more ways to minimize Sample size errors.