Point Estimation

An estimate is a number; an estimator is the rule

A parameter describes a population or model, such as mean μ or success probability p. An estimator is a function of random data, such as the sample mean. An estimate is the particular value obtained from the observed sample.

For hypothetical observations 2, 4, 6, the sample mean is 4. Another sample would generally produce another estimate. This uncertainty is why a point estimate should usually be accompanied by a standard error or interval and a description of the sampling design.

Worked example: sample variance and its denominator

For the same three values, deviations from 4 are −2, 0, 2 and the squared deviations sum to 8.

$s^2=\frac{\sum_i(x_i-\bar{x})^2}{n-1}=\frac8{2}=\mathbf4.$

Dividing by n instead gives 8/3 ≈ 2.6667. For independent identically distributed observations with finite variance, dividing by n − 1 makes the sample variance unbiased for the population variance. The mean was estimated from the same sample, which costs one degree of freedom.

That does not mean the realized estimate equals the true variance, nor that the sample standard deviation √s² is unbiased. Under a normal model, the maximum likelihood estimate of variance uses n rather than n − 1. Different goals can favor different rules.

Bias and variance measure repeated-sample behavior

Bias is E[θ̂] − θ. An estimator is unbiased when that difference is zero under its model. Variance measures how the estimator changes across samples. For squared-error loss:

$\operatorname{MSE}(\hat\theta)=\operatorname{Var}(\hat\theta)+\operatorname{Bias}(\hat\theta)^2.$

A slightly biased estimator can have lower mean squared error if it reduces variance enough. “Unbiased” is not synonymous with “best.” State the loss, model and comparison class when discussing efficiency.

Consistency means the estimator converges to the target as sample size grows under specified assumptions. It is not a promise that every successive sample estimate gets closer. Systematically sampling the wrong population can produce precise estimates of the wrong target.

Estimate a probability and its uncertainty

If 12 of 40 independent Bernoulli observations are successes, p̂ = 12/40 = 0.30. A plug-in standard error is √[0.30(0.70)/40] ≈ 0.0725. This simple approximation describes sampling uncertainty under the model, not bias from voluntary participation.

Near boundaries or with small counts, naive normal intervals can perform poorly; use an appropriate proportion interval rather than assuming the mean-confidence-interval calculator handles proportions. Observing no successes does not establish that the population probability is zero.

Maximum likelihood chooses parameters making the observed data most likely within a model. It does not mean the parameter itself has that probability, and it is not universally unbiased in finite samples.

Practice

An estimator has variance 4 and bias 2. What is its mean squared error?

Show the explanation

MSE = 4 + 2² = 8. Compare this with alternatives using the same target and squared-error criterion; checking bias alone would ignore sampling variability.

Continue

Read confidence intervals. NIST introduces maximum likelihood estimation.