How to Calculate Sample Size | How Many Survey Responses You Need, Why 95% and ±5% Gives 385, Tables by Population, Margin of Error, Confidence Intervals and A/B Test Sample Sizes

The short answer: 385 at 95% confidence and ±5%, fewer for a small population

The number of survey responses you need (the sample size) is fixed by two choices: how sure you want to be (the confidence level) and how many points of error you can accept (the margin of error). With the common 95% confidence level and ±5% margin of error, you need 385 responses when the population (everyone you want to learn about) is large enough. With 385 responses, a result of 40% “yes” means the true share is roughly between 35% and 45%.

Below, we first check where 385 comes from, then cover the correction for small populations, working back from responses to the margin of error and confidence interval, the response rate and group sizes, and finally A/B test significance and sample sizes. Every number here comes from the same calculation as the Sample Size Calculator.

The sample size formula: n = z² × p(1 − p) ÷ e² gives 385

The basic formula is n0 = z² × p(1 − p) ÷ e². Here z comes from the confidence level (1.96 for 95%), p is the expected proportion (roughly what share will answer “yes”), and e is the margin of error (0.05 for ±5%). When responses are collected at random, the spread of the observed proportion (its standard error) is √(p(1 − p) ÷ n), so the formula solves for the n at which z times that spread fits inside e.

1. 1

   Square z

   For 95% confidence z is 1.96, and 1.96² ≈ 3.8415.

2. 2

   Multiply by p(1 − p)

   With an expected proportion of 50%, 0.5 × 0.5 = 0.25, so 3.8415 × 0.25 ≈ 0.9604.

3. 3

   Divide by e²

   For ±5%, 0.05² = 0.0025, and 0.9604 ÷ 0.0025 ≈ 384.15.

4. 4

   Round up

   People come in whole numbers, so round up to 385. Rounding to the nearest would push the error slightly past the limit, so always round up.

Changing the confidence level changes z. The confidence level is the share of intervals that would contain the true value if you repeated the same survey many times. A higher level is more certain but needs more people.

Because you divide by e², the margin of error has a strong effect. Going from ±5% to ±2.5% (half) needs four times as many people, and ±1% needs 9,604, which is 25 times the ±5% figure. Shrinking the margin “to be as accurate as possible” quickly breaks the budget.

10% and 90% give the same number (if 10% say “yes”, 90% say “no”). A survey has many questions, each with its own proportion, so planning the overall size with 50% is the safe choice.

Finite population correction: a small population needs fewer responses

385 is the figure for a population large enough to treat as infinite. When the population is small, such as a company with 1,000 employees or a club with 500 members, you can reduce it with the finite population correction n = n0 ÷ (1 + (n0 − 1) ÷ N), where N is the population size.

1. 1

   Work out the infinite-population figure

   At 95%, ±5% and 50%, n0 ≈ 384.15 (use the value before rounding up).

2. 2

   Work out (n0 − 1) ÷ N

   For a population of 1,000, (384.15 − 1) ÷ 1,000 ≈ 0.3831.

3. 3

   Divide by 1 plus that value

   384.15 ÷ 1.3831 ≈ 277.7, which rounds up to 278. For a population of 10,000 it is 369.97, so 370.

As the table shows, once the population passes 100,000 the figure for ±5% barely moves, staying at 383 to 385. A national opinion poll and a survey of one city’s residents need almost the same number of people. What sets the sample size is not the share of the population you ask but the number of people who answer.

With a small population, on the other hand, the share becomes large. To reach ±5% in a department of 100 people, 80 of them, or 80%, need to answer. When the population is less than about 20 times the required sample (for ±5%, below roughly 7,700 people), entering the population size you know cuts the number noticeably.

Margin of error from the number of responses: ±4.9% for 400

If the responses are already in, you can turn the formula around and ask “what is the margin of error with this many people?”. The margin of error is e = z × √(p(1 − p) ÷ n), multiplied by √((N − n) ÷ (N − 1)) when you know the population N. For 400 responses at 95% and 50%, 1.96 × √(0.25 ÷ 400) = 1.96 × 0.025 ≈ ±4.9%.

The table is for 95% confidence and a large population. The ±9.8% for 100 people halves to ±4.9% for 400, and halving it again to around ±2.5% takes more than 1,500 people. The error shrinks with √n, so each extra respondent helps less and less. When a report says “margin of error up to ±x%”, use the 50% column, where the error is largest.

With a small population, the same number of responses gives a smaller error. If 400 out of a population of 1,000 answered, multiply by √((1,000 − 400) ÷ 999) ≈ 0.775 to get ±3.8%. If everyone answers (n = N), the error is zero for the group you surveyed. For 400 responses at other confidence levels, the error is ±4.1% at 90% and ±6.4% at 99%.

Confidence intervals: the Wald and Wilson methods

A confidence interval is the result with its margin of error attached. If 120 of 400 people (30%) said “yes”, the Wald method (normal approximation) gives 30% ± 1.96 × √(0.3 × 0.7 ÷ 400) ≈ 30% ± 4.5%, so the 95% confidence interval is 25.5% to 34.5%. This is the textbook formula, and it works well with a few hundred responses.

However, the Wald method gives odd intervals when there are few responses or when the proportion is close to 0% or 100%. The Wilson score method fixes this by pulling the centre slightly towards 50% and adjusting the width. The formula is a little more involved, so the tool shows both side by side.

With 400 people the two are almost identical, but with 3 of 30 the Wald lower limit goes negative and is cut off at 0%, and the upper limit comes out low. With 0 of 30, Wald says “0% to 0%”, as if there were no uncertainty at all. Wilson gives a sensible width: “even with zero, the true rate could be up to about 11%”. Use the Wilson method for small surveys and for proportions near 0%, such as defect or complaint rates.

Working back from the response rate, and sample sizes by group

The calculation gives the number of responses you need to collect. Not everyone answers a survey, so the number of people to invite is the required responses ÷ the response rate.

Response rates vary widely with how you distribute the survey and your relationship with the people you ask. An internal survey with a call to respond often gets more than half, while email surveys of customers may get only a few percent. Use your past response rate if you have it; if not, it is safer to assume a low one. With a population of 1,000 and a 30% response rate, you can invite at most 1,000 people and will get around 300 responses, only just above the 278 needed for ±5%.

If you want to compare results between groups, such as by gender or age, each group needs the required number of responses. For ±5% in each of five age bands you need 385 × 5 = 1,925 responses, which means inviting 6,417 people at a 30% response rate. If you collect only 385 in total, each age band has around 77 responses and an error of about ±11%.

The two-proportion test: is the A/B test difference just chance?

To compare proportions in an A/B test, or the share of “yes” between men and women or age groups, you use the two-proportion test (z-test). It takes the pooled proportion p̄ of both groups, computes the standard error of the difference √(p̄(1 − p̄) × (1/n1 + 1/n2)), and turns the number of standard errors in the difference (z) into a p-value. If the p-value is below the significance level (usually 5%), you conclude the difference is unlikely to be chance, i.e. it is significant.

1. 1

   Work out each proportion

   A: 100 clicks out of 1,000, so 10%. B: 130 out of 1,000, so 13%. The difference is 3 points (B is 1.3 times A).

2. 2

   Standard error from the pooled proportion

   p̄ = (100 + 130) ÷ 2,000 = 11.5%. √(0.115 × 0.885 × (1/1,000 + 1/1,000)) ≈ 0.01427.

3. 3

   Work out z and the p-value

   z = 0.03 ÷ 0.01427 ≈ 2.10. The two-sided p-value is about 0.035, below 5%, so the difference is significant.

The same 10% and 13% can lead to a different conclusion with a different number of people. Whether a difference is significant depends on how many people you compared, not just on its size. Looking at the 95% confidence interval of the difference (Wald, unpooled standard error) as well tells you how large the difference is likely to be.

With 300 each, the interval crosses zero, so B being worse cannot be ruled out. With 1,000 each, the whole interval is positive, but its lower end is +0.21 points: B is better, but the gap might be as small as 0.2 points. The p-value of this z-test equals that of a chi-square test on the 2×2 table without Yates’ correction.

Sample size for an A/B test: 1,774 per group for 10% → 13% at 80% power

The number of people you need so as not to miss a real difference depends on the current proportion p1, the proportion you hope for p2, the significance level and the power. Power is the probability of getting a significant result when a real difference exists, usually set to 80% or 90%. The sample size per group is n = (z_α/2 × √(2p̄(1 − p̄)) + z_β × √(p1(1 − p1) + p2(1 − p2)))² ÷ (p1 − p2)². At a 5% significance level (two-sided), z_α/2 = 1.96; z_β ≈ 0.842 for 80% power and ≈ 1.282 for 90%.

These figures are per group; the total for A and B is twice as much. For 10% → 13% you need 1,774 each, 3,548 in total. Shrinking the difference from 3 points to 2 more than doubles it to 3,841, and 1 point needs 14,751. The sample size is inversely proportional to the square of the difference, so confirming a small improvement takes a lot of traffic. Even for the same 5-point difference, 10% → 15% needs 686 while 50% → 55% needs 1,565: the closer the proportion is to 50%, the more people you need.

• With 200 visitors a day per variant, collecting 1,774 takes about 9 days. Decide the duration in advance, and run whole weeks so that weekday effects even out.
• Checking the p-value repeatedly and stopping the moment it drops below 5% makes a “significant” result likely even when there is no real difference. The rule is not to judge until you reach the planned sample size.
• Comparing many variants at once (B, C, D…) raises the chance that one of them is significant by luck. When there are many comparisons, use a stricter significance level (such as a Bonferroni correction) or test fewer variants.

When reality doesn’t match the formula: more people won’t remove bias

All the formulas above assume that the people who answer were chosen at random from the population. The margin of error covers only the random variation. If there is a bias in who answers, that error does not shrink even with thousands of responses.

• An online poll with 10,000 responses is not necessarily more accurate than a random sample of 400 if only people who want to answer take part.
• Significance shows only that a difference exists, not that it is large. With tens of thousands of people, even a 0.1-point difference is significant. Check the confidence interval of the difference to judge whether it matters.
• If the number of answers differs by question (some questions are optional), work out the error with each question’s own count.
• Free-text answers don’t lend themselves to percentages; summarise frequent words with something like a word cloud and report the counts alongside.

How to use the Sample Size Calculator

1. 1

   Get the responses you need

   In “Responses needed”, choose the confidence level (90%, 95% or 99%) and margin of error, and enter the population size (leave it blank if unknown). Leave the expected proportion at 50% if you don’t know it. The figures before the correction (infinite population) and after it are shown side by side.

2. 2

   Get invitations and group totals

   Enter the response rate to see how many people to invite, and the number of groups to compare to see the total responses needed.

3. 3

   Check the error of the responses you have

   In “Margin of error & CI”, enter the number of responses and the observed proportion to see the maximum margin of error and the confidence interval for that proportion (Wilson and Wald). Entering the population applies the finite population correction.

4. 4

   Compare two proportions

   In “Two proportions (A/B test)”, enter the number of people and the number of “yes” (clicks, purchases and so on) for A and B to see the difference, ratio, p-value, whether it is significant and the confidence interval of the difference. Below that, enter the current and hoped-for proportions and choose the power to get the sample size per group.

5. 5

   Get a rough idea from the charts

   “Charts” lists the required responses by population and margin of error, and the margin of error by number of responses and proportion.

All calculations run in your browser, and nothing you enter is sent anywhere. The numbers in this article come from the same calculations as the tool, so you can enter the examples to check them. Results can be copied as text or a URL and pasted straight into a survey plan or report.

FAQ

How many people need to answer my survey?

At 95% confidence and ±5% margin of error, 385 when the population is large. For a population of 10,000 it is 370, for 1,000 it is 278, for 500 it is 218 and for 100 it is 80: the smaller the population, the fewer you need. ±3% needs 1,068, and ±10% needs 97.

Where does the number 385 come from?

It is n = z² × p(1 − p) ÷ e² with z = 1.96 for 95% confidence, an expected proportion p = 0.5 and a margin of error e = 0.05. 1.96² × 0.25 ÷ 0.0025 ≈ 384.15, rounded up to 385.

Why set the expected proportion to 50%?

Because p(1 − p) is largest at 50%, which gives the largest sample size. If you plan with 50%, the error stays within your margin whatever proportion you get. If you expect around 10%, ±5% needs only 139, but if the guess is wrong the error grows.

What is the margin of error with 400 responses?

At 95% confidence, ±4.9% at most (when the result is 50%). It is ±9.8% for 100 and ±3.1% for 1,000. If 400 out of a population of 1,000 answered, the finite population correction brings it down to ±3.8%.

Do a population of 1 million and one of 10,000 need different sample sizes?

Hardly. At 95% and ±5%, a population of 1 million needs 384, 100,000 needs 383 and 10,000 needs 370. What sets the sample size is the number of respondents, not the share of the population, and the population only matters much below a few thousand.

Is the difference between a 10% and 13% click rate in an A/B test significant?

If A and B were each shown to 1,000 people and 100 and 130 clicked, the p-value is about 0.035, which is significant at the 5% level. The 95% confidence interval of the difference is +0.21 to +5.79 points. With the same rates but 300 people each, the p-value is about 0.25 and the difference is not significant.

How many people do I need for an A/B test?

To raise 10% to 13% at a 5% significance level and 80% power, you need 1,774 per group (3,548 in total). With 90% power it is 2,375; for a 2-point difference (10% → 12%) it is 3,841, and for 1 point 14,751. The smaller the difference, the faster the number climbs.

How do I get a confidence interval with only a few responses?

Use the Wilson score method. For 3 of 30 (10%), the 95% confidence interval is 3.5% to 25.6%. The common normal approximation (Wald) gives 0% to 20.7%, cut off at 0, and for 0 of 30 it gives 0% to 0% with no width at all, while Wilson gives a realistic 0% to 11.4%.