Sample Size Determination

Plan for precision or power, not an arbitrary number

“How many observations do I need?” has no universal answer. Decide whether the main goal is to estimate a quantity precisely or to detect a specified effect with a planned test. Define the population, outcome, independent unit, and design first.

An estimate of a mean, a binary proportion, a paired change, and a clustered regression model can require different planning. More rows do not necessarily mean more independent information.

Plan a mean estimate’s margin of error

For a simple independent sample, using a known or supplied planning SD σ and a normal critical value:

$n_0=\left(\frac{z_{1-\alpha/2}\sigma}{e}\right)^2$

Here e is the desired margin in the same units as the mean. Always round n₀ up to a whole number.

Suppose the desired 95% margin is 2 points and planning SD is 15. Using z = 1.96 gives:

$n_0=(1.96\times15/2)^2=216.09$

Plan at least 217 completed independent observations under this approximation. If σ comes from a pilot estimate, that estimate has uncertainty. Exact t-based planning or sensitivity analysis using several plausible SDs can be more informative than treating one pilot value as known.

Plan a proportion estimate

The common normal-approximation formula is:

$n_0=\frac{z_{1-\alpha/2}^2p(1-p)}{e^2}$

Use e as a proportion: ±5 percentage points means 0.05. Use an anticipated p from defensible prior information; p = 0.5 maximizes p(1 − p) when no better value is justified.

At 95% confidence, p = 0.5 and e = 0.05:

$n_0=1.96^2\times0.25/0.05^2=384.16$

Round up to 385. This is a planning approximation, not guaranteed realized precision for every sample. A rare outcome can require a different interval and planning method; using p = 0 or 1 to obtain n = 0 is not a defensible way to demonstrate certainty.

Precision changes quadratically

With other planning inputs fixed, n is proportional to 1/e². Halving the desired margin requires approximately four times the sample size. Doubling the planning SD also multiplies mean-estimation n by four.

Higher confidence requires a larger critical value and thus a larger n for the same planned margin. Distinguish a “higher confidence” plan from a “more precise” estimate; they are different goals.

Plan a balanced two-group mean test

For independent groups with common planning SD σ, target difference Δ, two-sided α, and power 1 − β, a normal approximation uses:

$n_{per\ group}\approx2\left(\frac{z_{1-\alpha/2}+z_{1-\beta}}{d}\right)^2,\qquad d=\frac{\Delta}{\sigma}$

With α = 0.05, power = 0.80, and d = 0.5, full-precision normal cutoffs give n ≈ 62.7910, rounded up to 63 per group. The site’s rounded cutoffs 1.96 and 0.84 give 62.72, also rounded to 63.

An exact t-based calculation can require a different n. The formula is not an exact guarantee and is not a paired-test, binary-outcome, unequal-allocation, or clustered-design calculation. Use a design-specific method when the simple balanced comparison is not your design.

Choose Δ for practical relevance in the original units before standardizing. A generic label such as “medium effect” should not replace a justified target.

Finite populations and sampling without replacement

For a simple random sample without replacement from a defined finite population of N units, a commonly used adjustment is:

$n'=\frac{n_0}{1+(n_0-1)/N}$

For n₀ = 384.16 and N = 1,000, n′ ≈ 277.7408, rounded up to 278. Use the unrounded initial calculation before the final rounding.

This adjustment reflects observing a material fraction of the actual finite population. It does not automatically apply to convenience samples, estimates about future populations, or clustered designs.

Allow for missing data and nonresponse

If 300 completed responses are needed and expected completion is 70%, invite at least ceiling(300/0.70) = 429 under that assumption. The planned invitation count is different from the completed-data requirement.

Inflating the number of invitations does not remove nonresponse bias. Consider which units are likely to be missing and how that affects the estimand and analysis. Do not silently call every invited person an analyzed observation.

Check sensitivity before committing

Repeat the plan over plausible values of variability, response rate, and target effect. If a modest change in assumptions doubles the requirement, communicate that uncertainty early.

Subgroup analyses, multiple comparisons, clustering, serial data, and missingness patterns can require more elaborate planning. Rules like “30 observations is enough” or “10 observations per predictor” do not guarantee valid inference or prediction.

Practice

A plan needs 385 completed responses. You expect 70% completion. How many invitations should you plan? If the proportion margin halves, what happens to the approximate completed-sample requirement?

Show the calculation

385/0.70 = 550 invitations. Halving the margin multiplies the unrounded sample calculation by four, assuming the same confidence, anticipated proportion, and design. Use the new formula and round once at the end.

References and tools

NIST’s mean-test planning reference connects sample size to α, power, variability, and target effect. The official statsmodels t-test power documentation illustrates a design-specific t-based alternative.

Use the sample size calculator for the simple approximations above and read effect size and power before planning a significance test.