ANOVA: Analysis of Variance

Compare several population means in one model

One-way analysis of variance (ANOVA) compares population means across independent groups defined by one categorical factor. For three workflows, its hypotheses are:

• H₀: μ₁ = μ₂ = μ₃.
• H₁: not all population means are equal.

The alternative does not say every group differs from every other group. A significant overall result needs further analysis to identify meaningful comparisons.

ANOVA tests means by comparing two estimates of variability: variation between group means and variation within groups.

Start with a plot and the study design

Before calculating, inspect group sizes, distributions, and unusual values. Ask whether the units were independently sampled or assigned. If the same person experiences all workflows, these are repeated measures; the independent-groups ANOVA below is not appropriate.

Classical one-way ANOVA assumes independent errors, common population error variance, and normally distributed errors for exact small-sample inference. It can tolerate some distributional departures, but unequal variances combined with unequal sample sizes can cause problems. Welch ANOVA relaxes the equal-variance assumption; rank methods answer different questions.

Neither an F-test nor a normality test can establish that the study design makes observations independent.

A complete example from nine measurements

Consider three hypothetical groups:

Group	Values	Group mean	Within-group sum of squares
A	2, 3, 4	3	1 + 0 + 1 = 2
B	4, 5, 6	5	2
C	6, 7, 8	7	2

There are k = 3 groups and N = 9 observations. The grand mean is 5. With unequal group sizes, calculate the grand mean from all observations or weight each group mean by its size; do not average the group means without weights.

1. Between-group sum of squares

$SS_B=\sum_{j=1}^k n_j(\bar x_j-\bar x)^2$

Here SS_B = 3(3 − 5)² + 3(5 − 5)² + 3(7 − 5)² = 24. This measures the spread of group means, accounting for how many observations each mean represents.

2. Within-group sum of squares

$SS_W=\sum_{j=1}^k\sum_{i=1}^{n_j}(x_{ij}-\bar x_j)^2$

Here SS_W = 2 + 2 + 2 = 6. Total SS is SS_B + SS_W = 30.

3. Divide by the relevant degrees of freedom

Source	SS	df	Mean square
Between groups	24	k − 1 = 2	24/2 = 12
Within groups	6	N − k = 6	6/6 = 1
Total	30	N − 1 = 8	—

The mean squares are variance estimates with different roles. Under H₀ and the model assumptions, their ratio is expected to be near 1, though it varies from sample to sample.

4. Form the F statistic

$F=\frac{MS_B}{MS_W}=\frac{12}{1}=12$

The reference distribution has numerator df = 2 and denominator df = 6. The 5% upper-tail critical value is 5.1433, and P(F₂,₆ ≥ 12) = 0.0080.

Reject equal population means at α = 0.05 under the classical model. A report could say: “The one-way ANOVA gave F(2, 6) = 12.00, p = 0.008; group sample means were 3, 5, and 7.” These tiny illustrative groups are useful for arithmetic, not for validating a model or making real-world claims.

Download the example observations (CSV)

What the overall test does not tell you

The F-test does not establish:

• Which particular pairs differ.
• That all pairs differ.
• That group membership causes the differences.
• That the differences are practically important.

For planned comparisons, use contrasts that address the original question. For all pairwise comparisons under the classical equal-variance model, a procedure such as Tukey’s can control familywise error. With unequal variances, consider procedures compatible with that model, such as Games–Howell after an appropriate analysis.

Do not run unadjusted pairwise tests repeatedly until a favorable contrast appears. Specify primary comparisons and distinguish confirmatory from exploratory results.

Report an effect estimate too

One descriptive effect-size measure is:

$\eta^2=\frac{SS_B}{SS_{Total}}$

Here η² = 24/30 = 0.80. In these data, 80% of the total corrected sum of squares lies between the group means. This is a sample measure; it is not proof that the factor causally explains 80% of population variation. η² can overestimate the population effect, especially in small samples; ω² is one alternative.

Report group means, variability, sizes, and intervals or contrast estimates in the original units. A standardized measure alone cannot tell a reader whether an operational change is worthwhile.

Read the F-table correctly

Use numerator df k − 1 and denominator df N − k, in that order. The F-table gives upper-tail critical values; reject the equal-means null for sufficiently large F.

The F distribution is not symmetric, so do not apply a ± cutoff. Do not halve α for this ANOVA test because the underlying mean differences can be positive or negative.

Practice

Four independent groups contain 24 observations in total. SS_B = 45 and SS_W = 60. Find df and F.

Show the calculation

Numerator df = 4 − 1 = 3; denominator df = 24 − 4 = 20. MS_B = 45/3 = 15; MS_W = 60/20 = 3. Thus F = 5. Compare with an upper-tail F distribution with df (3, 20), not (20, 3). The 5% critical value is about 3.0984.

References and next steps

NIST’s one-way ANOVA overview develops the sums-of-squares decomposition. Review t-tests for two means and effect size and power for design and reporting.