Statistics Glossary

Definitions with practical examples and common interpretation traps. Expand a term’s example to connect it to a real calculation.

A

Alpha Level (α)

A significance threshold chosen for a testing procedure. Under its null model and assumptions, it controls the long-run Type I error rate.

Example and interpretation

For a preplanned two-sided test at α = 0.05, a p-value of 0.04 crosses the threshold. It does not imply a 5% probability that this particular decision is wrong.

Related: significance levelType I ErrorP-Valuehypothesis testing
Category: Hypothesis Testing

Alternative Hypothesis (H₁)

A specified set of parameter values or models contrasted with the null hypothesis, such as a mean different from a target.

Example and interpretation

To ask whether mean package fill differs from 500 g, write H₁: μ ≠ 500 g. Choose a one-sided direction before inspecting the sample if that direction is justified.

Related: Null Hypothesishypothesis testingP-Value
Category: Hypothesis Testing

ANOVA (Analysis of Variance)

A family of methods comparing modeled sources of variation; one-way ANOVA tests equality of population means across independent groups.

Example and interpretation

For three groups and nine observations, a classical one-way ANOVA uses numerator df 2 and denominator df 6. A significant overall test does not identify which pairs differ.

Related: f-testVarianceMeanpost-hoc tests
Category: Inferential Statistics

B

Bar Chart

A graphical display using rectangular bars to represent categorical data, where bar heights or lengths show frequency or value.

Example and interpretation

A bar chart can compare counts of tea, coffee, and juice orders. Separate bars represent categories, rather than contiguous numerical intervals.

Category: Data Visualization

Bias

A systematic error in data collection, analysis, or interpretation that causes results to deviate from the true value in a particular direction.

Example and interpretation

A survey of only weekday customers may systematically miss weekend preferences. A larger weekday sample does not automatically remove that selection problem.

Related: sampling errorselection biasRandom Sampling
Category: Sampling

Binomial Distribution

A discrete probability distribution describing the number of successes in a fixed number of independent trials, each with the same probability of success.

Example and interpretation

With ten independent trials and success probability 0.2 on each, the expected success count is np = 2. Sampling without replacement from a small population changes the model.

Category: Probability Distributions

Box Plot

A graphical representation of data distribution showing the median, quartiles, and potential outliers through a five-number summary.

Example and interpretation

A box can show Q1 = 10, median = 15, and Q3 = 20. Whisker and outlier conventions vary, so identify the convention rather than assuming whiskers are always the minimum and maximum.

Related: QuartileMedianinterquartile rangeOutlier
Category: Data Visualization

C

Categorical Data

Values representing categories, which can be unordered (nominal) or ordered (ordinal). Numeric labels do not automatically make them quantitative.

Example and interpretation

Payment method is nominal; a satisfaction response from unhappy to happy is ordinal. Averaging numeric category codes is not automatically meaningful.

Related: Continuous DataDiscrete Datanominal dataordinal data
Category: Types of Data

Census

A complete count or survey of every member in a population, collecting data from all individuals rather than a sample.

Example and interpretation

Counting every enrolled student is a census of that school’s enrollment. Missing or incorrectly recorded students can still create error.

Related: PopulationSamplesampling
Category: Sampling

Central Limit Theorem

Under the usual independent, identically distributed, finite-variance conditions, the standardized sample mean approaches a normal distribution as sample size grows.

Example and interpretation

Averages of many independent die rolls become approximately normal even though individual rolls remain discrete. There is no universal sample-size cutoff that works for every population.

Category: Probability

Chi-Square (χ²)

A nonnegative probability distribution indexed by degrees of freedom, arising as a sum of squared independent standard normal variables.

Example and interpretation

If Z₁ and Z₂ are independent standard normal variables, Z₁² + Z₂² has a chi-square distribution with 2 df. Categorical tests use one application of this distribution.

Related: Categorical Datahypothesis testingDegrees of Freedom
Category: Inferential Statistics

Chi-Square Test

A family of procedures using a chi-square reference distribution, including categorical goodness-of-fit and independence tests.

Example and interpretation

A 2×3 independence table has df = (2−1)(3−1) = 2. Check expected counts and independent units before using the approximation.

Category: Inferential Statistics

Coefficient

A numerical multiplier or estimated model parameter; its units and interpretation depend on the model and variable coding.

Example and interpretation

In predicted score = 60 + 3 × hours, 3 is a slope coefficient with units of score points per hour. It describes a fitted association unless a causal interpretation is supported.

Related: CorrelationRegressionvariable
Category: Regression

Confidence Interval

An interval calculated by a procedure with a stated repeated-sampling coverage rate for a population parameter under its assumptions.

Example and interpretation

A 95% t interval for a mean uses an estimate plus or minus a margin. Across repetitions, its method has about 95% coverage under the assumptions; it does not contain 95% of individuals.

Category: Inferential Statistics

Continuous Data

Numerical data that can take any value within a range, including decimals and fractions, with infinite possible values between any two points.

Example and interpretation

A modeled waiting time can take any nonnegative real value, although a timer may record it rounded to seconds. The recording precision is not the same as the model’s support.

Related: Discrete DataCategorical Datainterval dataratio data
Category: Types of Data

Correlation

A measure of association. Pearson correlation describes linear association between two numerical variables and ranges from −1 to 1 when defined.

Example and interpretation

For paired X and Y with r = −0.8, the observed linear pattern decreases. A U-shaped relationship can have r = 0 despite strong dependence.

Related: CoefficientRegressionCovariancescatter plot
Category: Descriptive Statistics

Covariance

A measure of how two variables change together, indicating whether increases in one variable correspond to increases or decreases in another.

Example and interpretation

Positive covariance indicates that above-mean X tends to accompany above-mean Y. Its magnitude depends on units, unlike standardized Pearson correlation.

Category: Descriptive Statistics

D

Degrees of Freedom

The number of independent values that can vary in a statistical calculation, typically the sample size minus constraints applied.

Example and interpretation

A one-sample t procedure with 16 observations uses 15 df. A paired test counts pairs, and Welch’s independent-group df may be fractional.

Category: Inferential Statistics

Dependent Variable

The response or outcome modeled or measured in an analysis; the name alone does not establish a causal relationship.

Example and interpretation

When predicting waiting time from order size, waiting time is the response. Naming it dependent does not prove order size causes the observed differences.

Category: Regression

Descriptive Statistics

Statistical methods used to summarize and describe the main features of a dataset, including measures of central tendency and variability.

Example and interpretation

The mean of 2, 3, 3, 4, 8 is 4 and the median is 3. These describe the observed values without automatically generalizing to a population.

Category: Descriptive Statistics

Discrete Data

Numerical data that can only take specific, separate values, typically whole numbers that are counted rather than measured.

Example and interpretation

The number of orders in an hour is a count taking nonnegative integers. A numerical customer ID is a label, not a count variable.

Category: Types of Data

Distribution

The pattern or spread of data values across their possible range, showing how frequently different values occur.

Example and interpretation

Two datasets can share a mean while one has a long right tail. Plotting the distribution reveals what a single summary misses.

Category: Probability Distributions

E

Effect Size

A quantitative description of an effect or association, in original or standardized units. Its estimate has uncertainty and is not a p-value.

Example and interpretation

An observed 4-point mean difference is an effect estimate in original units. Dividing by a relevant SD gives a standardized measure whose definition should be labeled.

Category: Hypothesis Testing

Expected Value

The weighted average of all possible values of a random variable, where weights are the probabilities of each outcome occurring.

Example and interpretation

A fair six-sided die has expected value 3.5, although no single roll equals 3.5. Expectation is a probability-weighted mean, not the most likely outcome.

Related: MeanProbabilityrandom variable
Category: Probability

F

F-Distribution

A nonnegative distribution of the ratio of two independent chi-square variables divided by their respective degrees of freedom.

Example and interpretation

A classical ANOVA with three groups and nine observations uses F with df (2, 6). Swapping the numerator and denominator df changes the distribution.

Category: Probability Distributions

Frequency

The number of times a particular value or category occurs in a dataset, often displayed in frequency tables or histograms.

Example and interpretation

If coffee occurs in 30 of 50 orders, its frequency is 30 and relative frequency is 0.60. These are observed summaries, not guaranteed population probabilities.

Related: DistributionHistogramrelative frequency
Category: Descriptive Statistics

H

Histogram

A graphical representation of data distribution using adjacent bars to show the frequency of data within consecutive intervals or bins.

Example and interpretation

A histogram groups waits into intervals, such as 0–2 and 2–4 minutes. If bin widths differ, density heights rather than raw counts preserve area interpretation.

Category: Data Visualization

Hypothesis

A testable statement or prediction about the relationship between variables or characteristics of a population that can be evaluated using statistical methods.

Example and interpretation

A claim that population mean wait equals 10 minutes is a statistical hypothesis. Define the population and measurement before testing it.

Related: Null Hypothesisalternative hypothesishypothesis testing
Category: Hypothesis Testing

I

Independent Variable

A predictor or explanatory variable in a model, or a variable manipulated in an experiment. Its label does not imply statistical independence or causality.

Example and interpretation

In a model predicting score from study hours, hours is a predictor. It need not be independent of other predictors or randomly assigned.

Related: Dependent VariableRegressionvariableexperiment
Category: Regression

Inferential Statistics

Statistical methods that use sample data to draw conclusions, make predictions, or generalize findings to a larger population.

Example and interpretation

A sample mean and a confidence interval can estimate a population mean under a suitable design and model. Inference does not repair an unrepresentative sample.

Category: Inferential Statistics

Interquartile Range (IQR)

The difference between the third quartile (Q3) and first quartile (Q1), representing the middle 50% of data values.

Example and interpretation

If Q1 = 10 and Q3 = 18, the IQR is 8. Quartile algorithms can differ in small samples, so state the convention when reproducing results.

Category: Descriptive Statistics

K

Kurtosis

A measure of the 'tailedness' of a probability distribution, indicating how much data is in the tails compared to a normal distribution.

Example and interpretation

A normal distribution has kurtosis 3, or excess kurtosis 0. Different software can report either convention, and sample estimates are sensitive to extremes.

Category: Descriptive Statistics

L

Linear Regression

A model linear in its coefficients for a response as a function of predictors; simple regression fits a straight conditional-mean line.

Example and interpretation

A fitted line ŷ = 10 + 2x predicts a two-unit higher mean response for a one-unit higher predictor. An individual response still varies around the fitted mean.

Category: Regression

M

Margin of Error

The distance added to and subtracted from an estimate in a symmetric interval, often critical value times standard error. It is not a guaranteed error bound.

Example and interpretation

A mean interval 20 ± 2.13 has margin 2.13. It reflects the specified uncertainty procedure, not a maximum possible error across all sources.

Category: Inferential Statistics

Mean

The arithmetic average of a set of values, calculated by summing all values and dividing by the count.

Example and interpretation

For 2, 3, 3, 4, 8, the mean is 20/5 = 4. Combine group means by weighting with group sizes when the target is the individual-level mean.

Related: MedianModecentral tendency
Category: Descriptive Statistics

Median

A middle value of ordered data. For an even-sized numerical sample, the usual sample median averages the two middle observations.

Example and interpretation

For 2, 3, 4, 8, the median is (3+4)/2 = 3.5. Ties mean it is not always literally true that half the values are strictly below it.

Related: MeanModequartilespercentiles
Category: Descriptive Statistics

Mode

The most frequently occurring value in a dataset. A dataset can have one mode, multiple modes, or no mode.

Example and interpretation

For 2, 3, 3, 4, 8, the mode is 3. If every value is unique, courses often report no mode; a strict maximum-frequency definition instead gives a complete tie.

Related: MeanMedianfrequency distribution
Category: Descriptive Statistics

N

Normal Distribution

A symmetric, bell-shaped probability distribution where data clusters around the mean, with predictable proportions falling within standard deviations.

Example and interpretation

A central 95% normal interval uses approximately μ ± 1.96σ. The familiar μ ± 2σ interval contains about 95.45%, and only under a normal model.

Category: Probability Distributions

Null Hypothesis

A specified reference claim or model tested against an alternative, such as a population mean equal to a target value.

Example and interpretation

H₀: μ = 500 g is a target-value claim, not necessarily a claim of zero. A p-value is conditional on this model; it is not the model’s probability of being true.

Related: Alternative Hypothesis (H₁)P-Valuehypothesis-testingsignificance-level
Category: Hypothesis Testing

O

Outlier

An observation unusual relative to a distribution, model, or context. It may be a valid extreme observation or an error and requires investigation.

Example and interpretation

A wait of 60 minutes among mostly 2–5 minute waits merits investigation. Do not discard it merely because it changes the mean or p-value.

Related: interquartile rangeStandard DeviationBox Plot
Category: Descriptive Statistics

P

P-Value

The probability, under the null model and analysis assumptions, of a test statistic at least as extreme as the observed statistic.

Example and interpretation

If p = 0.03, results at least as extreme as observed have probability 3% under the null model and assumptions. This does not mean a 3% probability that the null is true.

Related: hypothesis-testingsignificance-levelNull Hypothesis
Category: Hypothesis Testing

Parameter

A numerical characteristic of a population, such as the population mean (μ) or population standard deviation (σ), typically estimated from sample data.

Example and interpretation

Population mean wait μ is a parameter; sample mean wait x̄ is a statistic used to estimate it. The target population must be defined.

Related: statisticPopulationSampleestimation
Category: Inferential Statistics

Percentile

A measure indicating the value below which a given percentage of observations fall, dividing data into 100 equal parts.

Example and interpretation

A 90th-percentile wait is a cutoff at or below which about 90% of the relevant distribution lies. Ties and sample quantile conventions affect exact empirical calculations.

Category: Descriptive Statistics

Population

The complete set of all individuals, objects, or measurements that share a common characteristic and are of interest for a statistical study.

Example and interpretation

For a café study, the population might be all eligible orders during one month. Changing the branch or period changes the population being described.

Category: Sampling

Probability

A numerical measure between 0 and 1 representing the likelihood that an event will occur, describing likelihood under a probability model. Probability zero and logical impossibility are not identical in continuous models.

Example and interpretation

A fair die assigns probability 1/6 to rolling a six. In a continuous model, an exact point can have probability zero without being logically impossible.

Related: Expected ValueDistributionrandom variable
Category: Probability

Q

Quartile

Values that divide a dataset into four equal parts, with Q1, Q2 (median), and Q3 representing the 25th, 50th, and 75th percentiles respectively.

Example and interpretation

Q2 is the median; Q1 and Q3 mark the lower and upper quarters. Small-sample interpolation or median-of-halves algorithms can give different results.

Related: PercentileMedianinterquartile rangeBox Plot
Category: Descriptive Statistics

R

Random Sampling

Selection using a specified probability mechanism. Simple random sampling gives equal inclusion probabilities; other probability designs need not.

Example and interpretation

In a simple random sample, eligible units have equal selection probabilities. Random selection does not guarantee every realized sample is perfectly representative.

Related: SamplePopulationBiassampling error
Category: Sampling

Range

The simplest measure of variability, calculated as the difference between the maximum and minimum values in a dataset.

Example and interpretation

For 2, 3, 3, 4, 8, the range is 8−2 = 6. It uses only the extremes and can change greatly with one unusual observation.

Related: VarianceStandard Deviationinterquartile range
Category: Descriptive Statistics

Regression

A statistical technique for modeling and analyzing the relationship between a dependent variable and one or more independent variables.

Example and interpretation

A regression model links a response to predictors. Check the functional form, dependence, variation, and influential observations before interpreting coefficients.

Category: Regression

Regression Analysis

A statistical method for modeling the relationship between a dependent variable and one or more independent variables, used for prediction and explanation.

Example and interpretation

Regression can estimate mean score as a function of hours and prior attainment. Adding predictors does not automatically remove confounding or improve out-of-sample prediction.

Category: Inferential Statistics

Residual

The observed response minus its fitted value. A residual is observable; the model error relative to the true mean function is generally not.

Example and interpretation

If an observed score is 88 and the fitted score is 86.7, its residual is 1.3. A residual is not the same as the unobserved population-model error.

Related: Regressionerrorpredicted value
Category: Regression

S

Sample

A subset of individuals or observations selected from a larger population for the purpose of statistical analysis and making inferences.

Example and interpretation

Eighty recorded orders can form a sample of a month’s eligible orders. A convenience sample may not represent the full month.

Related: PopulationsamplingstatisticRandom Sampling
Category: Sampling

Sample Size

The number of observations or data points collected from a population for analysis. Larger samples generally yield more precise estimates.

Example and interpretation

Forty people each measured twice give 40 pairs for a paired analysis, not 80 independent people. Precision depends on information and design, not just row count.

Category: Study Design

Sampling Distribution

The probability distribution of a statistic over repetitions of a specified sampling design and sample size.

Example and interpretation

Repeated independent samples of 25 observations produce a distribution of sample means. Its spread differs from the spread of individual observations.

Category: Inferential Statistics

Skewness

A measure of the asymmetry of a probability distribution, indicating whether data tails extend more to the left or right of the mean.

Example and interpretation

A long right tail often pulls the mean above the median. Mean–median ordering alone does not prove a distribution’s shape.

Category: Descriptive Statistics

Standard Deviation

A measure of the amount of variation or dispersion in a set of values, showing how spread out the data is from the mean.

Example and interpretation

For sample values 6, 8, 10, 12, 14, sample SD is √10 ≈ 3.1623. SD has the original units; variance has squared units.

Category: Descriptive Statistics

Standard Error

The standard deviation of a statistic’s sampling distribution, or an estimate of it. It describes sampling variability, not systematic bias.

Example and interpretation

With sample SD 10 and n = 25 independent observations, estimated SE of the mean is 2. SE measures sampling variability and does not include systematic bias.

Category: Inferential Statistics

Standard Normal Distribution

A normal distribution with a mean of 0 and a standard deviation of 1, used as the reference for z-scores and many statistical tests.

Example and interpretation

Z ~ N(0,1) has P(Z ≤ 1.96) ≈ 0.9750. Standardizing a skewed variable does not make it standard normal.

Category: Probability

Statistical Power

The probability that a testing procedure rejects its null under a specified alternative effect and model; it depends on design, sample size, variability, and threshold.

Example and interpretation

A design can target 80% power for a specified effect at a specified α. It does not have one universal power for every possible nonzero effect.

Category: Hypothesis Testing

Statistical Significance

A test result crossing a prechosen significance threshold under a specified procedure. It does not establish effect size, practical importance, or causality.

Example and interpretation

A preplanned test with p = 0.04 crosses α = 0.05 but not α = 0.01. Neither comparison says how large or practically valuable the effect is.

Related: P-Valuealpha levelhypothesis testingEffect Size
Category: Hypothesis Testing

T

t-Distribution

A symmetric distribution indexed by degrees of freedom, with heavier tails than normal at finite df; used in suitable inference with estimated variability.

Example and interpretation

For 15 df, a central 95% t interval uses cutoff 2.1314, larger than the normal cutoff 1.9600. Unknown population SD does not become known when n exceeds 30.

Category: Probability Distributions

t-Test

A statistical test used to compare means between one or two groups, determining whether differences are statistically significant.

Example and interpretation

A paired t-test analyzes within-unit differences; Welch’s t-test compares independent means. Same-length columns alone do not justify pairing.

Related: t-Distributionhypothesis testingMeanP-Value
Category: Hypothesis Testing

Type I Error

Rejecting a null hypothesis when that null is true.

Example and interpretation

Rejecting a true claim that μ = 500 g is a Type I error. α controls a procedure’s rate under its null assumptions, not the probability a specific rejection is wrong.

Related: Type II Erroralpha levelhypothesis testingP-Value
Category: Hypothesis Testing

Type II Error

Failing to reject a null hypothesis when a specified alternative is true. Failure to reject does not mean concluding that no effect exists.

Example and interpretation

Failing to reject μ = 500 g when the true mean is 495 g is a Type II error for that alternative. A nonsignificant result does not prove no difference.

Related: Type I ErrorStatistical Powerhypothesis testingbeta
Category: Hypothesis Testing

V

Variance

The mean squared deviation from the mean for a population, or a sample-based estimate of it, typically using n−1 when estimating population variance.

Example and interpretation

The sample squared-deviation sum for 6, 8, 10, 12, 14 is 40; sample variance is 40/4 = 10. Describing the complete five-value population instead gives 40/5 = 8.

Related: Standard DeviationMeanRangedispersion
Category: Descriptive Statistics

Z

Z-Score

A standardized measure indicating how many standard deviations a data point is from the mean, used for comparing values across different distributions.

Example and interpretation

A value 62 relative to mean 50 and SD 8 has z = 1.5. Its normal upper-tail probability is 0.0668 only if the normal model is justified.

Category: Descriptive Statistics