Which test should I use?
The best method depends on what you measure and how you collected it. Choose your question below, then check the assumptions before calculating.
One-sample t-test
For independent numerical observations with unknown population standard deviation. Check for severe skew and outliers, especially with a small sample.
Welch’s two-sample t-test
Use for two unrelated groups with numerical outcomes. Welch’s method does not require equal variances. Independence and suitable distribution assumptions still matter.
Paired t-test
Calculate the difference within each pair, then test the mean difference. Pairs must be independent of one another; assess the distribution of the differences.
One-way ANOVA
For a numerical outcome grouped by one categorical factor. The standard ANOVA model assumes independent observations, roughly normal errors, and equal variances. Consider Welch’s ANOVA if variances differ.
Chi-square test of independence
Use counts in a contingency table, with independent observations and sufficient expected cell counts. Sparse tables may need an exact test. Use goodness-of-fit instead when comparing one categorical distribution to specified proportions.
Pearson correlation
For paired numerical variables with a roughly linear relationship. Inspect a scatterplot and influential outliers. The usual significance test assumes independent pairs and appropriate distribution conditions; correlation does not establish causation.
Don’t let sample size choose the test for you
“Use z when n ≥ 30” is too simple. Inference about a mean with population standard deviation estimated from your sample generally uses t. A large sample makes t and z cutoffs closer; it doesn’t make the population standard deviation known.
Decide before looking at the result
State your null and alternative hypotheses, choose a significance level, and decide whether a directional test is justified before analyzing the data. Trying multiple tests until one gives p < 0.05 changes the error rate.
Report more than a p-value
Include the estimated effect, an uncertainty interval when appropriate, the sample size, and the test assumptions. A statistically significant result can be practically small; a non-significant result does not prove there is no effect.
Feedback on optional paid features
Would you use either of these features?
We’re exploring optional paid extras while keeping these tools free. Would either be useful to you at these prices?
An interest check only. Nothing to buy or sign up for.