Logistic Regression Explained: How to Run, Read and Report It
Logistic regression models the probability of a binary outcome, such as pass or fail, relapse or no relapse, from one or more predictors. Instead of predicting the outcome directly, it models the log-odds, so predicted probabilities always lie between 0 and 1. This guide explains the method and a fully worked example.
What is logistic regression?
For a binary outcome with probability π, the logit model is:
ln[π/(1 − π)] = β0 + β1x1 + … + βkxk π = 1 / (1 + e−(β0 + β1x))
Coefficients are estimated by maximum likelihood, not least squares. The exponentiated slope, eβ, is the odds ratio: the multiplicative change in the odds of the outcome per one-unit increase in the predictor. An odds ratio is not a risk ratio, and the two diverge when the outcome is common.
When should you use it?
Use it for a binary outcome with continuous or categorical predictors, and extend it to multinomial or ordinal versions when the outcome has more categories. A frequently cited guide is at least about 10 outcome events per predictor (Peduzzi et al., 1996), though more recent work suggests that this is only a rough rule and that stability depends on context.
Assumptions
- Independent observations and a binary outcome with correctly specified categories.
- Linearity of the logit for continuous predictors (check with a polynomial or spline term or a Box-Tidwell-style test).
- No severe multicollinearity and no perfect or quasi-perfect separation, which makes maximum-likelihood estimates diverge.
- Adequate number of events per predictor.
Original example (illustrative)
Question. Do weekly study hours predict passing an exam? H0: β1 = 0. Data for n = 20 students (hours → pass = 1, fail = 0): 1h→0; 2h→0; 2h→0; 3h→0; 3h→1; 4h→0; 4h→0; 5h→1; 5h→0; 6h→0; 6h→1; 7h→1; 7h→0; 8h→1; 8h→1; 9h→1; 9h→1; 10h→1; 11h→1; 12h→1. There are 11 passes and 9 fails.
Maximum-likelihood estimates (computed by iteratively reweighted least squares): b0 = −4.112 (SE 1.866), b1 = 0.749 (SE 0.317). The Wald test for the slope gives z = 2.37, p = .018. The odds ratio is e0.749 = 2.12, 95% CI [1.14, 3.94] (Wald). Log-likelihoods are −13.76 (null) and −7.91 (model), so the likelihood-ratio statistic is χ²(1) = 2 × (−7.91 − (−13.76)) = 11.71, p < .001. McFadden's pseudo-R² = 1 − −7.91/−13.76 = 0.43 (McFadden, 1974). Predicted probability of passing is 41% at 5 hours and 87% at 8 hours. Classifying at a 0.50 cut-off, 80% of students are classified correctly (an in-sample figure that overstates real predictive accuracy).
Interpretation. Each additional weekly study hour multiplies the odds of passing by about 2.12, i.e. roughly a 112% increase in the odds. The interval is wide because n = 20, so the direction is clear but the size is uncertain. Do not read the odds ratio as a 112% increase in the probability.
How to run logistic regression with StatFAS
Open the Logistic Regression Calculator. Arrange the data with a binary outcome column coded 0/1 (or two clear labels) and one or more predictor columns.
Understanding the output
- Coefficients and Wald tests: direction and significance on the log-odds scale.
- Odds ratios with CIs: the main effect size; an interval including 1 indicates no clear association.
- Likelihood-ratio test: compares the model with the null model, and is usually preferred over Wald tests in small samples.
- Pseudo-R²: not interpretable as variance explained.
- Classification table / ROC-AUC: predictive performance, ideally on new data.
APA 7 reporting example
A logistic regression indicated that weekly study hours significantly predicted exam success, χ²(1) = 11.71, p < .001, McFadden R² = .43. Each additional hour was associated with higher odds of passing (OR = 2.12, 95% CI [1.14, 3.94]).
Common mistakes and troubleshooting
- Reporting odds ratios as risk ratios or percentage-point changes.
- Perfect separation: the software reports huge coefficients or fails to converge. Consider penalised (Firth) logistic regression or fewer predictors.
- Too few events for the number of predictors.
- Dichotomising continuous predictors unnecessarily, which discards information.
- Judging fit only by accuracy when classes are imbalanced.
Limitations and alternatives
Logistic regression estimates association; confounding and selection can distort odds ratios. For a continuous outcome use linear regression; for counts use negative binomial regression; for time-to-event outcomes use Cox regression. For a simple binary proportion test see the binomial test.
Conclusion
Logistic regression is the standard tool for binary outcomes, and its results are most useful when reported as odds ratios with confidence intervals, together with model fit and a realistic view of sample size.
1.
statfas-logistic-regression-interface.png, alt: "StatFAS Logistic Regression Calculator interface"2.
statfas-logistic-regression-example-data.png, alt: "Example data entered in the StatFAS Logistic Regression Calculator"3.
statfas-logistic-regression-settings.png, alt: "Logistic Regression Calculator settings in StatFAS"4.
statfas-logistic-regression-result.png, alt: "Logistic Regression Calculator results generated in StatFAS"Frequently asked questions
What does an odds ratio of 1.5 mean?
It means the odds of the outcome are 1.5 times higher for each one-unit increase in the predictor (or compared with the reference category), not that the probability is 1.5 times higher.
What is the difference between odds and probability?
Probability is the chance of the outcome, p. Odds are p/(1 − p). They are close when the outcome is rare but diverge as it becomes common.
Why do I get a convergence warning?
Common causes are perfect separation, too few events for the number of predictors, or highly collinear predictors. Simplify the model or use a penalised method such as Firth regression.
References
- Hosmer, D. W., Lemeshow, S., & Sturdivant, R. X. (2013). Applied logistic regression (3rd ed.). Wiley.
- Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996). A simulation study of the number of events per variable in logistic regression analysis. Journal of Clinical Epidemiology, 49(12), 1373–1379.
- McFadden, D. (1974). Conditional logit analysis of qualitative choice behavior. In P. Zarembka (Ed.), Frontiers in econometrics (pp. 105–142). Academic Press.