Hypothesis Testing: A Beginner's Complete Guide

Turn an analysis into a report a reader can assess

A good hypothesis-test report makes the question, estimate, uncertainty, model, and limitations visible. A statement such as “p < 0.05, therefore it works” leaves almost all of that out.

This guide focuses on reporting. The hypothesis-testing lesson explains the mechanics and a full calculation.

Begin with the question and unit

Suppose 16 independently sampled packs have mean fill 498 g and sample SD 4 g. The target is 500 g. State the population, sampling period, and whether a row represents a pack, a production batch, or a repeated measurement.

Define H₀: μ = 500 g and H₁: μ ≠ 500 g, with α = 0.05 chosen before examining the result. Population SD is unknown, so the illustrative calculation uses a one-sample t model.

Show the result in original units

SE = 4/√16 = 1 g. The mean difference from target is −2 g. The t statistic is −2 with 15 df, giving two-sided p ≈ 0.0639.

The 95% t interval for μ is [495.8686, 500.1314] g. These quantities tell the reader more than the significance label alone: the estimate is below target, and the uncertainty includes both the target and appreciable underfilling.

Use a reporting template

In 16 independently sampled packs, mean fill was 498 g (sample SD 4 g). The estimated difference from the 500 g target was −2 g. The 95% t interval for the population mean was 495.87–500.13 g; a two-sided one-sample test gave t(15) = −2.00, p = 0.0639. The data did not reject the target mean at the predeclared 5% threshold, but the interval remains compatible with potentially important underfilling.

This is an original report for hypothetical data. In a real report, add how the sample was collected, model checks, exclusions, and the operational tolerance that defines a meaningful difference.

Distinguish evidence from decisions

The p-value is conditional on H₀ and the analysis assumptions. It is not the probability H₀ is true, the probability the result is “just chance,” or an effect-size measure.

A threshold decision can support a planned testing procedure, but practical action also depends on effect magnitude, uncertainty, costs, and design. A small p-value does not demonstrate causation; a large p-value does not establish equality.

The ASA statement on p-values explicitly separates significance from effect size and warns against drawing conclusions only from a threshold.

Explain what was planned and what was explored

If you tried several outcomes, subgroups, transformations, exclusions, or stopping points, disclose them. Selecting only a favorable test result can distort the reader’s view of the evidence. Preplanned and exploratory analyses can both be useful when labeled accurately.

For multiple comparisons or repeated checking, use a procedure designed for that analysis. Do not attach a fixed-sample interpretation to a test stopped at the first favorable p-value.

A final review checklist

• Population, outcome, and independent unit are defined.
• Sample size, estimate, and measurement units are included.
• Method, alternative, df where relevant, and p-value method are stated.
• Interval and practical meaning accompany the test.
• Missing data, exclusions, assumptions, and limitations are visible.
• Causal claims match the design; nonsignificance is not called equivalence.

Use the t-test calculator for raw-data examples, the confidence interval calculator for summaries, and the test-selection guide when the design is still unclear.