Regression Analysis Explained: How to Choose, Run and Report the Right Model
"Regression" is not one test but a family of models that relate an outcome to one or more predictors. The single most useful decision is choosing the right member of the family, and that choice is driven mainly by what your outcome variable is. This guide gives a practical decision framework, explains what all regression models share, and shows how to move from a research question to the correct StatFAS regression tool.
What all regression models have in common
Every regression model has the form g(E[y]) = β0 + β1x1 + … + βkxk, where g is a link function that matches the outcome's scale. For a continuous outcome, g is the identity; for a binary outcome, the logit; for counts, the log. Coefficients are estimated by least squares or maximum likelihood, and each comes with a standard error, a confidence interval and a test. Interpretation differs by model: a linear slope is a difference in means, a logistic slope is a log-odds ratio, a negative binomial slope is a log rate ratio and a Cox slope is a log hazard ratio.
Choosing a regression model: the outcome decides
| Outcome type | Example | Typical model | Effect measure |
|---|---|---|---|
| Continuous, roughly symmetric residuals | Exam score | Linear / multiple linear | Mean difference |
| Continuous, curved relationship | Growth curve | Polynomial | Curve shape |
| Continuous, skewed or outlier-prone; interest in quantiles | Income | Quantile | Quantile difference |
| Binary | Relapse yes/no | Logistic | Odds ratio |
| Overdispersed counts | Number of infections | Negative binomial | Rate ratio |
| Time to event, with censoring | Survival time | Cox | Hazard ratio |
Then ask three further questions. How many predictors, and how correlated are they? Many correlated predictors point to ridge or lasso regression. Does theory order the predictors? Entering them in blocks calls for hierarchical regression. Is your question about how or when an effect operates? That is mediation or moderation analysis, not a different outcome type.
Original decision walk-through (illustrative)
A clinical researcher has 120 patients and asks four questions. (1) Does weekly exercise predict systolic blood pressure at 12 weeks? The outcome is continuous, so use linear or multiple linear regression, adjusting for age and baseline pressure. (2) Does exercise predict whether the patient is hypertensive at 12 weeks (yes/no)? Binary outcome, so logistic regression and report an odds ratio. (3) Does exercise predict the number of hospital visits in a year? Counts with variance larger than the mean point to negative binomial regression, and the effect is a rate ratio. (4) Does exercise predict time until first cardiac event, where some patients are still event-free at the end of follow-up? That requires Cox regression because censoring makes ordinary regression invalid. One dataset, four research questions, four models: the outcome, not the predictors, decided each choice.
Assumptions to check, by model family
- Linear: linearity, independence, constant variance, approximately normal residuals, no influential outliers.
- Logistic: independence, linear logit for continuous predictors, enough events per predictor, no separation.
- Count models: independence; check overdispersion (variance much larger than mean) and zero inflation.
- Cox: proportional hazards, independent censoring.
- All: correct model specification, independent observations (repeated or clustered data need multilevel or GEE approaches) and limited multicollinearity.
How to start in StatFAS
Open the Regression Calculator. If you are unsure which test fits your question, StatFAS also lists a hypothesis analysis suggester.
Reporting regression results
Whatever the model, report the sample size, the model type and link, each coefficient with its confidence interval, the effect measure on its natural scale (difference, odds ratio, rate ratio, hazard ratio), an overall fit or test statistic, and the checks you made. For example: A multiple linear regression (n = 120) adjusted for age and baseline pressure estimated that each additional hour of weekly exercise was associated with a 1.2 mmHg lower systolic pressure (95% CI [−2.0, −0.4]). That sentence is an illustration of the format, not a result.
Common mistakes
- Using linear regression for a binary or count outcome. Predictions leave the valid range and errors are not normal.
- Dichotomising a continuous outcome to use logistic regression, which loses information.
- Ignoring censoring in time-to-event data.
- Treating adjusted coefficients as causal. Regression adjusts for measured variables only.
- Overfitting: too many predictors for the sample size or number of events.
- Stepwise selection producing biased, unstable models; see stepwise regression and its limitations.
Limitations and alternatives
Regression estimates conditional associations under a chosen model. If the form is wrong, results mislead; use residual diagnostics and consider flexible alternatives (splines, generalised additive models) when relationships are curved. For simple group comparisons, a t-test or ANOVA is the same linear model in a special case.
Conclusion
Pick the regression by the outcome, refine it by the structure of your predictors and question, and report effects on their natural scale with confidence intervals. Once the model is chosen, a calculator is simply a convenient way to run it.
1.
statfas-regression-calculator-interface.png, alt: "StatFAS Regression Calculator interface"2.
statfas-regression-calculator-example-data.png, alt: "Example data entered in the StatFAS Regression Calculator"3.
statfas-regression-calculator-settings.png, alt: "Regression Calculator settings in StatFAS"4.
statfas-regression-calculator-result.png, alt: "Regression Calculator results generated in StatFAS"Frequently asked questions
Is a t-test a form of regression?
Yes. A two-group t-test is equivalent to a linear regression with one binary predictor, and ANOVA is equivalent to a linear regression with categorical predictors.
How do I choose between logistic and linear regression?
Use logistic regression for a binary outcome and linear regression for a continuous outcome. Using linear regression for a binary outcome can give impossible predictions and misleading standard errors.
How many predictors can I include in a regression model?
There is no fixed limit. Sample size, effect size, outcome type (events for binary or survival outcomes) and desired precision all matter, and overfitting becomes likely when predictors approach the number of observations or events.
References
- Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Erlbaum.
- Hosmer, D. W., Lemeshow, S., & Sturdivant, R. X. (2013). Applied logistic regression (3rd ed.). Wiley.
- Cox, D. R. (1972). Regression models and life-tables. Journal of the Royal Statistical Society: Series B, 34(2), 187–220.