How many people need to answer a survey?

With 95% confidence and a ±5% margin of error, 385 for a large population. For a population of 10,000 it is 370, for 1,000 it is 278, for 500 it is 218 and for 100 it is 80, so smaller populations need fewer. At ±3% you need 1,068, and at ±1% 9,604.

What is the difference between 95% and 99% confidence?

It is how often the true value would fall inside the interval if you repeated the same survey many times. 99% is more certain but needs more people. For ±5%, you need 271 at 90%, 385 at 95% and 664 at 99% (for a large population).

Is a 10% vs 13% click rate in an A/B test significant?

If A and B were each shown to 1,000 people and 100 and 130 clicked, the p-value is about 0.035, which is significant at the 5% level. The 95% confidence interval of the difference is +0.21 to +5.79 points. With the same rates but 300 people each, the p-value is about 0.25 and the difference is not significant.

How do I decide the sample size for an A/B test?

From the current rate, the rate you expect, the significance level and the power. To raise 10% to 13% at a 5% significance level and 80% power, you need 1,774 per group (3,548 in total). For a smaller difference such as 10% vs 12%, it rises to 3,841.

What if I do not know the population size?

Leave the population blank and it is treated as large (infinite). For populations of 100,000 or more the result hardly changes: 383 to 385 for 95% and ±5%. If the population is less than about 20 times the number of responses, entering an approximate figure reduces the number needed.

Sample Size Calculator | Survey Responses Needed (Population, Confidence Level, Margin of Error, Finite Population Correction, Number to Invite from the Response Rate), Margin of Error and Confidence Interval (Wilson) from the Number of Responses, Two-Proportion Test (A/B Test Significance and p-Value) and Sample Size for a Difference (Power), Charts of Required Sample Size and Margin of Error

The Sample Size Calculator brings together the headcount and error calculations for surveys, such as “how many of our 1,000 employees need to answer?”, “what is the margin of error with 400 responses?” and “is the difference between a 10% and a 13% click rate in an A/B test significant?”. In the required responses tab, it uses n0 = z² × p(1 − p) ÷ e² with the finite population correction to give the minimum number of responses from the population size (leave it blank if unknown), confidence level (90%, 95% or 99%), margin of error and expected proportion (385 for 95% and ±5%, or 370 for a population of 10,000). It also works out how many people to invite from the response rate and the total needed when you analyse several groups such as age bands. In the margin of error tab, it gives the margin of error for the responses you collected (±4.9% for 400) and the confidence interval for your result by the Wilson score method and the normal approximation (Wald). In the two-proportion tab, it runs a two-proportion z-test (two-sided) for A/B tests or comparisons between men and women or age groups, showing the p-value, whether the difference is significant and the confidence interval of the difference, and calculates the sample size per group needed to detect a difference (80% or 90% power). It includes a chart of required responses by population and margin of error, a chart of margin of error by number of responses and proportion, and explanations of confidence level, finite population correction, p-value, power and other terms.

Survey responses needed (sample size)

Find the minimum number of responses for reliable results from the population size, confidence level and margin of error.

Confidence level

Margin of error (±)

Population sizepeopleMargin of error%Expected proportion%Response rate%Groups to comparegroups

Responses needed

370

After finite population correction

For a large population

385

z = 1.960

Formula: n0 = z² × p(1 − p) ÷ e²; when the population N is known, n = n0 ÷ (1 + (n0 − 1) ÷ N). If you cannot predict the proportion, 50% is the safe choice because it needs the most responses.

These calculations assume responses from randomly selected people. Recruiting on social media or surveys answered only by people who want to answer are biased, so more responses will not make them as accurate as the error calculation suggests.

ABOUT

About Sample Size Calculator | Survey Responses Needed (Population, Confidence Level, Margin of Error, Finite Population Correction, Number to Invite from the Response Rate), Margin of Error and Confidence Interval (Wilson) from the Number of Responses, Two-Proportion Test (A/B Test Significance and p-Value) and Sample Size for a Difference (Power), Charts of Required Sample Size and Margin of Error

The Sample Size Calculator works out how many responses a survey needs from the population size, confidence level and margin of error. It also gives the margin of error and confidence interval for the responses you collected, and tests whether the difference between two proportions, such as in an A/B test or between men and women, is significant.

How to use Sample Size Calculator | Survey Responses Needed (Population, Confidence Level, Margin of Error, Finite Population Correction, Number to Invite from the Response Rate), Margin of Error and Confidence Interval (Wilson) from the Number of Responses, Two-Proportion Test (A/B Test Significance and p-Value) and Sample Size for a Difference (Power), Charts of Required Sample Size and Margin of Error

1In “Responses needed”, choose the confidence level (usually 95%) and margin of error (such as ±5%)
2Enter the population size (number of employees, members and so on). Leave it blank if you do not know it
3Enter the response rate to see how many people to invite. If you will analyse groups such as age bands separately, enter the number of groups
4Once responses are in, enter the number of responses and the result in “Margin of error & CI” to check the error and confidence interval
5To compare two groups or an A/B test, enter the number of people and the number of “yes” in “Two proportions” and check the p-value and significance

Great for

Deciding the minimum number of responses for an internal survey, customer satisfaction survey or thesis questionnaire
Checking the margin of error for each sample size before ordering an online panel survey, to balance budget and accuracy
Stating in a report how many points the results could be off, given the responses collected
Checking whether a difference in click rate or conversion rate in an A/B test is more than chance
Checking whether the share of “yes” differs between men and women or between age groups
Estimating the traffic needed to detect a difference before starting an A/B test

Good to know

The required number of responses is n0 = z² × p(1 − p) ÷ e² (z is 1.96 for 95% confidence, p is the expected proportion and e is the margin of error). When the population N is known, the finite population correction n = n0 ÷ (1 + (n0 − 1) ÷ N) is applied and the result is rounded up.
If you have no idea of the expected proportion, use 50%, which needs the most responses; the error will then stay within the margin whatever the result.
Confidence intervals are shown by both the Wilson score method and the normal approximation (Wald). With few responses or a proportion near 0% or 100%, the Wilson method is more accurate.
The two-proportion test is a pooled z-test (two-sided), which gives the same p-value as a 2×2 chi-square test without Yates' correction. The sample size uses the normal-approximation formula for a two-sided test with 80% or 90% power.
The calculations assume responses from randomly selected people. Recruiting on social media or surveys answered only by people who want to answer are biased, so more responses will not make them as accurate as the error calculation suggests.

Questions this tool answers

Use Sample Size Calculator | Survey Responses Needed (Population, Confidence Level, Margin of Error, Finite Population Correction, Number to Invite from the Response Rate), Margin of Error and Confidence Interval (Wilson) from the Number of Responses, Two-Proportion Test (A/B Test Significance and p-Value) and Sample Size for a Difference (Power), Charts of Required Sample Size and Margin of Error for searches like these:

Find how many survey responses are needed
Calculate a sample size
Calculate the margin of error from the number of responses
Get a confidence interval for survey results
Check whether an A/B test result is significant
Find the sample size needed for an A/B test

FAQ

How many people need to answer a survey?: With 95% confidence and a ±5% margin of error, 385 for a large population. For a population of 10,000 it is 370, for 1,000 it is 278, for 500 it is 218 and for 100 it is 80, so smaller populations need fewer. At ±3% you need 1,068, and at ±1% 9,604.
What is the margin of error with 400 responses?: At 95% confidence, ±4.9% at most (when the result is 50%). With 100 responses it is ±9.8%, and with 1,000 it is ±3.1%. If 400 out of a population of 1,000 answered, the finite population correction brings it down to ±3.8%.
What is the difference between 95% and 99% confidence?: It is how often the true value would fall inside the interval if you repeated the same survey many times. 99% is more certain but needs more people. For ±5%, you need 271 at 90%, 385 at 95% and 664 at 99% (for a large population).
Is a 10% vs 13% click rate in an A/B test significant?: If A and B were each shown to 1,000 people and 100 and 130 clicked, the p-value is about 0.035, which is significant at the 5% level. The 95% confidence interval of the difference is +0.21 to +5.79 points. With the same rates but 300 people each, the p-value is about 0.25 and the difference is not significant.
How do I decide the sample size for an A/B test?: From the current rate, the rate you expect, the significance level and the power. To raise 10% to 13% at a 5% significance level and 80% power, you need 1,774 per group (3,548 in total). For a smaller difference such as 10% vs 12%, it rises to 3,841.
What if I do not know the population size?: Leave the population blank and it is treated as large (infinite). For populations of 100,000 or more the result hardly changes: 383 to 385 for 95% and ±5%. If the population is less than about 20 times the number of responses, entering an approximate figure reduces the number needed.

Special Thanks

SOLO GUILD | サービス作るAIエンジニア

Back to all tools