Logistic Regression

Model a binary outcome through its probability

Logistic regression models an outcome coded 0 or 1, such as completion within a fixed time window. With predictors x, the model relates a linear predictor η to the probability p of the outcome coded 1:

$\eta=\beta_0+\beta_1x,\qquad \log\frac{p}{1-p}=\eta,\qquad p=\frac1{1+e^{-\eta}}.$

It is linear in log odds, not in probability. A predictor change can have a different probability effect at different starting values. Normal residuals are not a requirement for this binomial model.

Worked example: hypothetical fitted coefficients

Suppose an illustrative model for completing a tutorial is η = −2 + 0.5x, where x is the number of previous lessons. These coefficients are supplied for calculation practice, not estimates from a real study.

Previous lessons x	Log odds η	Predicted probability
0	−2	0.1192
2	−1	0.2689
4	0	0.5000
6	1	0.7311

For a one-lesson increase, the model odds multiply by e⁰·⁵ ≈ 1.6487, with any other predictors held fixed. This is a 64.9% increase in odds, not a 64.9% increase in probability and not a 64.9 percentage-point increase.

From x = 2 to 3, probability rises from 0.2689 to 0.3775, about 10.86 percentage points. From x = 4 to 5, it rises from 0.5000 to 0.6225, about 12.25 points. An odds ratio is constant in this model while the probability difference is not.

Fitting and checking the model

Coefficients are commonly fitted by maximizing the Bernoulli likelihood. With an interaction or transformed predictor, interpret the complete expression rather than one coefficient in isolation.

Check whether observations are independent or whether repeat users, families or sites require a model for clustering. Evaluate whether the log-odds relationship is adequately represented, whether influential observations dominate, and whether predictors contain redundant information.

Complete separation—where a predictor perfectly distinguishes outcomes—can prevent a finite ordinary maximum likelihood estimate. Small datasets and rare outcomes can also create unstable estimates. These are reasons to investigate fitting and regularization, not to declare a perfect model.

Prediction quality is more than accuracy

Assess calibration: do predicted probabilities agree with observed frequencies? Assess discrimination: can the model rank outcomes? Use held-out or otherwise appropriately validated data, and avoid leaking future information into predictors.

A classification threshold such as 0.5 is a decision choice, not part of the probability model. Its usefulness depends on outcome frequency and the consequences of errors. Report uncertainty and population context. An adjusted association does not establish that completing an extra lesson causes the predicted increase.

Practice

A coefficient is ln(2). Does a one-unit increase double probability?

Show the explanation

It doubles the model odds, with other predictors held fixed. If initial probability is 0.2, odds are 0.25; doubling gives odds 0.5 and probability 0.5/1.5 = 1/3, not 0.4.

Continue

Review multiple regression and conditional probability. Penn State’s logistic regression lesson explains logits, odds and coefficient interpretation.