Chi-Square Tests

Compare counts with a model for those counts

A chi-square test compares observed category counts with the counts expected under a null model. Two common versions answer different questions:

Version	Question	Null model
Goodness of fit	Does one categorical variable follow specified proportions?	The category probabilities equal specified values
Independence	Are two categorical variables associated?	Their joint probabilities factor into row and column probabilities

Use counts of independent units, not means, arbitrary scores, or percentages detached from sample sizes. A table of purchases can contain repeated purchases by the same customer; it is not automatically a table of independent people.

The statistic measures departures from expected counts

$\chi^2=\sum\frac{(O-E)^2}{E}$

O is an observed count; E is its expected count under H₀. Calculate one contribution per category or cell and add them. Expected counts may be fractional; do not round them to whole people before calculating.

Large discrepancies increase χ². For these categorical tests, the p-value is the upper-tail probability P(χ²_df ≥ observed χ²). Do not double it because an association could go in different directions: the squaring already combines departures of either sign.

Worked goodness-of-fit example

A fair four-category spinner is spun independently 80 times. Counts are 30, 20, 18, 12. Under equal category probabilities, each expected count is 80 × 0.25 = 20.

Category	Observed	Expected	Contribution
A	30	20	100/20 = 5
B	20	20	0
C	18	20	4/20 = 0.2
D	12	20	64/20 = 3.2
Total	80	80	8.4

With four fixed probabilities, df = k − 1 = 3. At α = 0.05, the chi-square table gives critical value 7.8147. Since 8.4 > 7.8147, reject the specified equal-probability model; p ≈ 0.0384.

The result concerns compatibility with that model. It does not tell you why the proportions differ, establish dishonesty, or identify one particular category as the only source of departure.

If the null distribution has parameters estimated from the same counts, degrees of freedom may need adjustment. Under the usual suitable conditions, subtract the number m of estimated parameters: df = k − 1 − m. Distribution fitting and binning require care; see NIST’s goodness-of-fit reference.

Worked test of independence

A hypothetical survey classifies 100 independent respondents by route and preference:

Route	Prefer option A	Prefer option B	Total
North	30	20	50
South	20	30	50
Total	50	50	100

Under independence:

$E_{ij}=\frac{(\text{row total}_i)(\text{column total}_j)}{n}$

Every cell has E = 50 × 50/100 = 25. Each contributes (5²)/25 = 1, so χ² = 4. Degrees of freedom are (r − 1)(c − 1) = 1.

The uncorrected Pearson test gives p ≈ 0.0455; its 5% critical value is 3.8415. This indicates evidence of an association at the chosen threshold, under the test assumptions.

These numbers use no continuity correction. Software may default to Yates’ correction for a 2×2 table; that produces a different statistic and p-value. Report which version you used instead of mixing one method’s statistic with another method’s p-value.

The observed preference rates are 60% and 40%. That 20-percentage-point difference describes the sample. The test does not show that route causes preference.

NIST’s contingency-table reference explains expected counts and the independence test.

Check whether the approximation is appropriate

• Each unit belongs in one cell of the table and should be independent of other units.
• Categories should be mutually exclusive and meaningful for the question.
• Expected counts must be large enough for the chi-square approximation. Requiring all E ≥ 5 is a useful conservative introductory check; more nuanced guidance exists for larger tables.
• Check expected, not observed, counts. A large total n does not guarantee adequate counts in rare categories.

For sparse 2×2 tables, Fisher’s exact test may be suitable. Larger sparse tables may need exact or Monte Carlo procedures. Merge categories only when substantively defensible, not merely to obtain a desired result.

Paired binary outcomes require a paired method such as McNemar’s test. Repeatedly measuring the same respondents violates the independent-table model.

Explain the size and pattern of association

For an r×c table, Cramér’s V is:

$V=\sqrt{\frac{\chi^2}{n\min(r-1,c-1)}}$

For the example, V = √(4/100) = 0.20. Its practical meaning depends on the setting and table dimensions. Report the actual category proportions alongside it; do not treat universal “weak/moderate/strong” bands as a decision rule.

Pearson residuals (O − E)/√E can help locate departures. Their magnitudes are diagnostic clues, not automatically independent significance tests. Searching across many cells needs suitable multiplicity handling, and adjusted residuals use a different denominator.

Practice

A goodness-of-fit test compares counts 18, 22, 20 with equal probabilities. Calculate χ² and df.

Show the calculation

The total is 60, so each expected count is 20. χ² = 4/20 + 4/20 + 0 = 0.4. With three fully specified categories, df = 2. The upper-tail p-value is about 0.8187, so the data do not give strong evidence against equal probabilities. This does not prove equality.

Continue learning

Use the chi-square table for critical values and degrees of freedom reference to distinguish designs. For a binary outcome with predictors, continue to logistic regression.