Statistics Glossary
Definitions with practical examples and common interpretation traps. Expand a term’s example to connect it to a real calculation.
A
Alpha Level (α)
A significance threshold chosen for a testing procedure. Under its null model and assumptions, it controls the long-run Type I error rate.
Example and interpretation
For a preplanned two-sided test at α = 0.05, a p-value of 0.04 crosses the threshold. It does not imply a 5% probability that this particular decision is wrong.
Alternative Hypothesis (H₁)
A specified set of parameter values or models contrasted with the null hypothesis, such as a mean different from a target.
Example and interpretation
To ask whether mean package fill differs from 500 g, write H₁: μ ≠ 500 g. Choose a one-sided direction before inspecting the sample if that direction is justified.
ANOVA (Analysis of Variance)
A family of methods comparing modeled sources of variation; one-way ANOVA tests equality of population means across independent groups.
Example and interpretation
For three groups and nine observations, a classical one-way ANOVA uses numerator df 2 and denominator df 6. A significant overall test does not identify which pairs differ.
B
Bar Chart
A graphical display using rectangular bars to represent categorical data, where bar heights or lengths show frequency or value.
Example and interpretation
A bar chart can compare counts of tea, coffee, and juice orders. Separate bars represent categories, rather than contiguous numerical intervals.
Bias
A systematic error in data collection, analysis, or interpretation that causes results to deviate from the true value in a particular direction.
Example and interpretation
A survey of only weekday customers may systematically miss weekend preferences. A larger weekday sample does not automatically remove that selection problem.
Binomial Distribution
A discrete probability distribution describing the number of successes in a fixed number of independent trials, each with the same probability of success.
Example and interpretation
With ten independent trials and success probability 0.2 on each, the expected success count is np = 2. Sampling without replacement from a small population changes the model.
Box Plot
A graphical representation of data distribution showing the median, quartiles, and potential outliers through a five-number summary.
Example and interpretation
A box can show Q1 = 10, median = 15, and Q3 = 20. Whisker and outlier conventions vary, so identify the convention rather than assuming whiskers are always the minimum and maximum.
C
Categorical Data
Values representing categories, which can be unordered (nominal) or ordered (ordinal). Numeric labels do not automatically make them quantitative.
Example and interpretation
Payment method is nominal; a satisfaction response from unhappy to happy is ordinal. Averaging numeric category codes is not automatically meaningful.
Census
A complete count or survey of every member in a population, collecting data from all individuals rather than a sample.
Example and interpretation
Counting every enrolled student is a census of that school’s enrollment. Missing or incorrectly recorded students can still create error.
Central Limit Theorem
Under the usual independent, identically distributed, finite-variance conditions, the standardized sample mean approaches a normal distribution as sample size grows.
Example and interpretation
Averages of many independent die rolls become approximately normal even though individual rolls remain discrete. There is no universal sample-size cutoff that works for every population.
Chi-Square (χ²)
A nonnegative probability distribution indexed by degrees of freedom, arising as a sum of squared independent standard normal variables.
Example and interpretation
If Z₁ and Z₂ are independent standard normal variables, Z₁² + Z₂² has a chi-square distribution with 2 df. Categorical tests use one application of this distribution.
Chi-Square Test
A family of procedures using a chi-square reference distribution, including categorical goodness-of-fit and independence tests.
Example and interpretation
A 2×3 independence table has df = (2−1)(3−1) = 2. Check expected counts and independent units before using the approximation.
Coefficient
A numerical multiplier or estimated model parameter; its units and interpretation depend on the model and variable coding.
Example and interpretation
In predicted score = 60 + 3 × hours, 3 is a slope coefficient with units of score points per hour. It describes a fitted association unless a causal interpretation is supported.
Confidence Interval
An interval calculated by a procedure with a stated repeated-sampling coverage rate for a population parameter under its assumptions.
Example and interpretation
A 95% t interval for a mean uses an estimate plus or minus a margin. Across repetitions, its method has about 95% coverage under the assumptions; it does not contain 95% of individuals.
Continuous Data
Numerical data that can take any value within a range, including decimals and fractions, with infinite possible values between any two points.
Example and interpretation
A modeled waiting time can take any nonnegative real value, although a timer may record it rounded to seconds. The recording precision is not the same as the model’s support.
Correlation
A measure of association. Pearson correlation describes linear association between two numerical variables and ranges from −1 to 1 when defined.
Example and interpretation
For paired X and Y with r = −0.8, the observed linear pattern decreases. A U-shaped relationship can have r = 0 despite strong dependence.
Covariance
A measure of how two variables change together, indicating whether increases in one variable correspond to increases or decreases in another.
Example and interpretation
Positive covariance indicates that above-mean X tends to accompany above-mean Y. Its magnitude depends on units, unlike standardized Pearson correlation.
D
Degrees of Freedom
The number of independent values that can vary in a statistical calculation, typically the sample size minus constraints applied.
Example and interpretation
A one-sample t procedure with 16 observations uses 15 df. A paired test counts pairs, and Welch’s independent-group df may be fractional.
Dependent Variable
The response or outcome modeled or measured in an analysis; the name alone does not establish a causal relationship.
Example and interpretation
When predicting waiting time from order size, waiting time is the response. Naming it dependent does not prove order size causes the observed differences.
Descriptive Statistics
Statistical methods used to summarize and describe the main features of a dataset, including measures of central tendency and variability.
Example and interpretation
The mean of 2, 3, 3, 4, 8 is 4 and the median is 3. These describe the observed values without automatically generalizing to a population.
Discrete Data
Numerical data that can only take specific, separate values, typically whole numbers that are counted rather than measured.
Example and interpretation
The number of orders in an hour is a count taking nonnegative integers. A numerical customer ID is a label, not a count variable.
Distribution
The pattern or spread of data values across their possible range, showing how frequently different values occur.
Example and interpretation
Two datasets can share a mean while one has a long right tail. Plotting the distribution reveals what a single summary misses.
E
Effect Size
A quantitative description of an effect or association, in original or standardized units. Its estimate has uncertainty and is not a p-value.
Example and interpretation
An observed 4-point mean difference is an effect estimate in original units. Dividing by a relevant SD gives a standardized measure whose definition should be labeled.
Expected Value
The weighted average of all possible values of a random variable, where weights are the probabilities of each outcome occurring.
Example and interpretation
A fair six-sided die has expected value 3.5, although no single roll equals 3.5. Expectation is a probability-weighted mean, not the most likely outcome.
F
F-Distribution
A nonnegative distribution of the ratio of two independent chi-square variables divided by their respective degrees of freedom.
Example and interpretation
A classical ANOVA with three groups and nine observations uses F with df (2, 6). Swapping the numerator and denominator df changes the distribution.
Frequency
The number of times a particular value or category occurs in a dataset, often displayed in frequency tables or histograms.
Example and interpretation
If coffee occurs in 30 of 50 orders, its frequency is 30 and relative frequency is 0.60. These are observed summaries, not guaranteed population probabilities.
H
Histogram
A graphical representation of data distribution using adjacent bars to show the frequency of data within consecutive intervals or bins.
Example and interpretation
A histogram groups waits into intervals, such as 0–2 and 2–4 minutes. If bin widths differ, density heights rather than raw counts preserve area interpretation.
Hypothesis
A testable statement or prediction about the relationship between variables or characteristics of a population that can be evaluated using statistical methods.
Example and interpretation
A claim that population mean wait equals 10 minutes is a statistical hypothesis. Define the population and measurement before testing it.
I
Independent Variable
A predictor or explanatory variable in a model, or a variable manipulated in an experiment. Its label does not imply statistical independence or causality.
Example and interpretation
In a model predicting score from study hours, hours is a predictor. It need not be independent of other predictors or randomly assigned.
Inferential Statistics
Statistical methods that use sample data to draw conclusions, make predictions, or generalize findings to a larger population.
Example and interpretation
A sample mean and a confidence interval can estimate a population mean under a suitable design and model. Inference does not repair an unrepresentative sample.
Interquartile Range (IQR)
The difference between the third quartile (Q3) and first quartile (Q1), representing the middle 50% of data values.
Example and interpretation
If Q1 = 10 and Q3 = 18, the IQR is 8. Quartile algorithms can differ in small samples, so state the convention when reproducing results.
K
Kurtosis
A measure of the 'tailedness' of a probability distribution, indicating how much data is in the tails compared to a normal distribution.
Example and interpretation
A normal distribution has kurtosis 3, or excess kurtosis 0. Different software can report either convention, and sample estimates are sensitive to extremes.
L
Linear Regression
A model linear in its coefficients for a response as a function of predictors; simple regression fits a straight conditional-mean line.
Example and interpretation
A fitted line ŷ = 10 + 2x predicts a two-unit higher mean response for a one-unit higher predictor. An individual response still varies around the fitted mean.
M
Margin of Error
The distance added to and subtracted from an estimate in a symmetric interval, often critical value times standard error. It is not a guaranteed error bound.
Example and interpretation
A mean interval 20 ± 2.13 has margin 2.13. It reflects the specified uncertainty procedure, not a maximum possible error across all sources.
Mean
The arithmetic average of a set of values, calculated by summing all values and dividing by the count.
Example and interpretation
For 2, 3, 3, 4, 8, the mean is 20/5 = 4. Combine group means by weighting with group sizes when the target is the individual-level mean.
Median
A middle value of ordered data. For an even-sized numerical sample, the usual sample median averages the two middle observations.
Example and interpretation
For 2, 3, 4, 8, the median is (3+4)/2 = 3.5. Ties mean it is not always literally true that half the values are strictly below it.
Mode
The most frequently occurring value in a dataset. A dataset can have one mode, multiple modes, or no mode.
Example and interpretation
For 2, 3, 3, 4, 8, the mode is 3. If every value is unique, courses often report no mode; a strict maximum-frequency definition instead gives a complete tie.
N
Normal Distribution
A symmetric, bell-shaped probability distribution where data clusters around the mean, with predictable proportions falling within standard deviations.
Example and interpretation
A central 95% normal interval uses approximately μ ± 1.96σ. The familiar μ ± 2σ interval contains about 95.45%, and only under a normal model.
Null Hypothesis
A specified reference claim or model tested against an alternative, such as a population mean equal to a target value.
Example and interpretation
H₀: μ = 500 g is a target-value claim, not necessarily a claim of zero. A p-value is conditional on this model; it is not the model’s probability of being true.
O
Outlier
An observation unusual relative to a distribution, model, or context. It may be a valid extreme observation or an error and requires investigation.
Example and interpretation
A wait of 60 minutes among mostly 2–5 minute waits merits investigation. Do not discard it merely because it changes the mean or p-value.
P
P-Value
The probability, under the null model and analysis assumptions, of a test statistic at least as extreme as the observed statistic.
Example and interpretation
If p = 0.03, results at least as extreme as observed have probability 3% under the null model and assumptions. This does not mean a 3% probability that the null is true.
Parameter
A numerical characteristic of a population, such as the population mean (μ) or population standard deviation (σ), typically estimated from sample data.
Example and interpretation
Population mean wait μ is a parameter; sample mean wait x̄ is a statistic used to estimate it. The target population must be defined.
Percentile
A measure indicating the value below which a given percentage of observations fall, dividing data into 100 equal parts.
Example and interpretation
A 90th-percentile wait is a cutoff at or below which about 90% of the relevant distribution lies. Ties and sample quantile conventions affect exact empirical calculations.
Population
The complete set of all individuals, objects, or measurements that share a common characteristic and are of interest for a statistical study.
Example and interpretation
For a café study, the population might be all eligible orders during one month. Changing the branch or period changes the population being described.
Probability
A numerical measure between 0 and 1 representing the likelihood that an event will occur, describing likelihood under a probability model. Probability zero and logical impossibility are not identical in continuous models.
Example and interpretation
A fair die assigns probability 1/6 to rolling a six. In a continuous model, an exact point can have probability zero without being logically impossible.
Q
Quartile
Values that divide a dataset into four equal parts, with Q1, Q2 (median), and Q3 representing the 25th, 50th, and 75th percentiles respectively.
Example and interpretation
Q2 is the median; Q1 and Q3 mark the lower and upper quarters. Small-sample interpolation or median-of-halves algorithms can give different results.
R
Random Sampling
Selection using a specified probability mechanism. Simple random sampling gives equal inclusion probabilities; other probability designs need not.
Example and interpretation
In a simple random sample, eligible units have equal selection probabilities. Random selection does not guarantee every realized sample is perfectly representative.
Range
The simplest measure of variability, calculated as the difference between the maximum and minimum values in a dataset.
Example and interpretation
For 2, 3, 3, 4, 8, the range is 8−2 = 6. It uses only the extremes and can change greatly with one unusual observation.
Regression
A statistical technique for modeling and analyzing the relationship between a dependent variable and one or more independent variables.
Example and interpretation
A regression model links a response to predictors. Check the functional form, dependence, variation, and influential observations before interpreting coefficients.
Regression Analysis
A statistical method for modeling the relationship between a dependent variable and one or more independent variables, used for prediction and explanation.
Example and interpretation
Regression can estimate mean score as a function of hours and prior attainment. Adding predictors does not automatically remove confounding or improve out-of-sample prediction.
Residual
The observed response minus its fitted value. A residual is observable; the model error relative to the true mean function is generally not.
Example and interpretation
If an observed score is 88 and the fitted score is 86.7, its residual is 1.3. A residual is not the same as the unobserved population-model error.
S
Sample
A subset of individuals or observations selected from a larger population for the purpose of statistical analysis and making inferences.
Example and interpretation
Eighty recorded orders can form a sample of a month’s eligible orders. A convenience sample may not represent the full month.
Sample Size
The number of observations or data points collected from a population for analysis. Larger samples generally yield more precise estimates.
Example and interpretation
Forty people each measured twice give 40 pairs for a paired analysis, not 80 independent people. Precision depends on information and design, not just row count.
Sampling Distribution
The probability distribution of a statistic over repetitions of a specified sampling design and sample size.
Example and interpretation
Repeated independent samples of 25 observations produce a distribution of sample means. Its spread differs from the spread of individual observations.
Skewness
A measure of the asymmetry of a probability distribution, indicating whether data tails extend more to the left or right of the mean.
Example and interpretation
A long right tail often pulls the mean above the median. Mean–median ordering alone does not prove a distribution’s shape.
Standard Deviation
A measure of the amount of variation or dispersion in a set of values, showing how spread out the data is from the mean.
Example and interpretation
For sample values 6, 8, 10, 12, 14, sample SD is √10 ≈ 3.1623. SD has the original units; variance has squared units.
Standard Error
The standard deviation of a statistic’s sampling distribution, or an estimate of it. It describes sampling variability, not systematic bias.
Example and interpretation
With sample SD 10 and n = 25 independent observations, estimated SE of the mean is 2. SE measures sampling variability and does not include systematic bias.
Standard Normal Distribution
A normal distribution with a mean of 0 and a standard deviation of 1, used as the reference for z-scores and many statistical tests.
Example and interpretation
Z ~ N(0,1) has P(Z ≤ 1.96) ≈ 0.9750. Standardizing a skewed variable does not make it standard normal.
Statistical Power
The probability that a testing procedure rejects its null under a specified alternative effect and model; it depends on design, sample size, variability, and threshold.
Example and interpretation
A design can target 80% power for a specified effect at a specified α. It does not have one universal power for every possible nonzero effect.
Statistical Significance
A test result crossing a prechosen significance threshold under a specified procedure. It does not establish effect size, practical importance, or causality.
Example and interpretation
A preplanned test with p = 0.04 crosses α = 0.05 but not α = 0.01. Neither comparison says how large or practically valuable the effect is.
T
t-Distribution
A symmetric distribution indexed by degrees of freedom, with heavier tails than normal at finite df; used in suitable inference with estimated variability.
Example and interpretation
For 15 df, a central 95% t interval uses cutoff 2.1314, larger than the normal cutoff 1.9600. Unknown population SD does not become known when n exceeds 30.
t-Test
A statistical test used to compare means between one or two groups, determining whether differences are statistically significant.
Example and interpretation
A paired t-test analyzes within-unit differences; Welch’s t-test compares independent means. Same-length columns alone do not justify pairing.
Type I Error
Rejecting a null hypothesis when that null is true.
Example and interpretation
Rejecting a true claim that μ = 500 g is a Type I error. α controls a procedure’s rate under its null assumptions, not the probability a specific rejection is wrong.
Type II Error
Failing to reject a null hypothesis when a specified alternative is true. Failure to reject does not mean concluding that no effect exists.
Example and interpretation
Failing to reject μ = 500 g when the true mean is 495 g is a Type II error for that alternative. A nonsignificant result does not prove no difference.
V
Variance
The mean squared deviation from the mean for a population, or a sample-based estimate of it, typically using n−1 when estimating population variance.
Example and interpretation
The sample squared-deviation sum for 6, 8, 10, 12, 14 is 40; sample variance is 40/4 = 10. Describing the complete five-value population instead gives 40/5 = 8.
Z
Z-Score
A standardized measure indicating how many standard deviations a data point is from the mean, used for comparing values across different distributions.
Example and interpretation
A value 62 relative to mean 50 and SD 8 has z = 1.5. Its normal upper-tail probability is 0.0668 only if the normal model is justified.