Sampling Distributions

Three distributions that are easy to confuse

Suppose a café’s order waits have population mean μ = 10 minutes and SD σ = 4 minutes. Draw 25 orders and compute their mean. Draw another 25 orders and compute another mean. The means vary across samples.

Distribution	What its values represent	Spread
Population distribution	Waits of all eligible individual orders	Population SD σ
Sample distribution	The individual waits in one sample	Sample SD s
Sampling distribution of x̄	Means from repeated samples of the same size	Standard error of x̄

A sampling distribution describes a statistic, not the original observations. You normally see only one sample; a probability model or simulation helps describe the repeated-sampling behavior.

Center and spread of a sample mean

For n independent observations from the same population, with finite mean and variance:

$E(\bar X)=\mu,\qquad\operatorname{Var}(\bar X)=\frac{\sigma^2}{n},\qquad SE(\bar X)=\frac{\sigma}{\sqrt n}$

The mean is centered at the population mean. Its standard error shrinks as sample size grows. These center-and-spread facts do not require a normal population.

For the café model:

Sample size	SD of individual waits	SE of the sample mean
4	4 minutes	2 minutes
25	4 minutes	0.8 minutes
100	4 minutes	0.4 minutes

More observations do not make individual waits shorter or less variable. They make the estimate of the mean more precise, if the sampling conditions remain comparable.

When σ is unknown, s/√n estimates the standard error. This estimate inherits the limitations of the data and sampling design.

What the central limit theorem adds

For independent identically distributed observations with finite variance, the standardized sample mean approaches standard normal as n grows:

$\frac{\bar X-\mu}{\sigma/\sqrt n}\;\xrightarrow{d}\;N(0,1)$

If the population is normal, the sample mean is exactly normal for any n under independent sampling. If the population is not normal, the normal approximation generally improves with n, but the needed size depends on skew and tail behavior.

“n ≥ 30” is a classroom rule of thumb, not a guarantee. Very heavy-tailed populations can converge slowly; populations without finite variance fall outside this elementary version of the theorem. The theorem also does not repair bias or dependence.

Worked example: an individual versus a mean

Assume individual waits follow a normal model with μ = 10 and σ = 4. What is the probability of exceeding 11.6 minutes?

For one order:

$z=\frac{11.6-10}{4}=0.4$

P(X > 11.6) = 1 − Φ(0.4) ≈ 0.3446.

For a mean of 25 independent orders:

$SE=\frac4{\sqrt{25}}=0.8,\qquad z=\frac{11.6-10}{0.8}=2$

P(x̄ > 11.6) = 1 − Φ(2) ≈ 0.0228.

A mean that far from μ is less common than an individual wait that far from μ. Using σ instead of SE would answer the wrong question.

For nonnormal waits, the first probability requires their actual distribution; the second might use a justified normal approximation. You cannot apply the empirical rule to individual waits merely because the sample is large.

A simulation you can run in a spreadsheet

A fair six-sided die has mean 3.5 and variance 35/12.

Generate six independent die rolls and calculate their mean.
Repeat 1,000 times, keeping each six-roll mean as one value.
Plot a histogram of the 1,000 means.
Repeat using 36 rolls per mean.

Both histograms should center near 3.5. Their theoretical SEs are √(35/12)/√6 ≈ 0.6972 and √(35/12)/√36 ≈ 0.2846. The 36-roll means should cluster more tightly and look more bell-shaped.

Individual die rolls remain discrete and uniformly distributed on 1 through 6. Increasing the number of rolls per mean changes the distribution of the mean, not the die.

Proportions have a sampling distribution too

For n independent Bernoulli trials with fixed success probability p:

$E(\hat p)=p,\qquad SE(\hat p)=\sqrt{\frac{p(1-p)}n}$

A common normal-approximation check requires both np and n(1 − p) to be sufficiently large, often at least 10 in introductory work. Use the null proportion when checking a null-model test; interval methods make their own assumptions. With rare events or small samples, use suitable binomial methods instead of relying on a normal approximation.

Dependence changes the calculation

Repeated observations of the same person, students in one classroom, and measurements over time can be correlated. Positive correlation often makes s/√n too optimistic because n observations contain less information than n independent observations.

For a simple random sample without replacement from a finite population of size N, the usual independent-draw SE based on a population variance with denominator N is multiplied by:

$\sqrt{\frac{N-n}{N-1}}$

This finite population correction reflects the extra information gained by observing a substantial fraction of that population. It does not apply automatically to convenience samples or clustered designs.

Practice

A population has SD 12. Compare the SE of the mean for n = 9 and n = 36. Does the population SD change?

Show the explanation

The SEs are 4 and 2. Quadrupling the sample size halves the SE under independent sampling. The population SD remains 12; collecting more data does not alter the population’s spread.

Continue learning

NIST’s normal-distribution discussion explains its connection to sampling means. Next, use sampling variability to construct confidence intervals and understand hypothesis tests.