Multiple Linear Regression Explained: How to Run, Read and Report It
Multiple linear regression models a continuous outcome from two or more predictors at once. Its main value is that each coefficient is adjusted for the other predictors, so you can ask what a predictor adds after accounting for the rest. This guide explains the method, the output, and a fully worked two-predictor example.
What is multiple linear regression?
The model is y = β0 + β1x1 + β2x2 + … + βkxk + ε, estimated by least squares. In matrix form the estimates are b = (X′X)−1X′y. Each coefficient is the expected change in y for a one-unit increase in that predictor holding the other predictors constant, so it can differ from the coefficient in a simple regression.
When should you use it?
Use it to adjust for confounders, to compare the contribution of several predictors, or to build a prediction equation for a continuous outcome. It needs more observations than predictors; a commonly cited guide is at least 10 to 20 cases per predictor, but adequate sample size depends on the expected effect size and desired precision (Cohen et al., 2003).
Mathematical foundation
R² = SSreg/SStotal adj. R² = 1 − (1 − R²)(n − 1)/(n − k − 1) F = (SSreg/k)/(SSres/(n − k − 1))
Each coefficient is tested by t = bj/SE(bj) with n − k − 1 degrees of freedom. The variance inflation factor, VIFj = 1/(1 − Rj²), where Rj² comes from regressing predictor j on the other predictors, measures collinearity. Standardised coefficients, βj = bj(sxj/sy), put predictors on a common scale.
Assumptions
- Linear relationships between predictors and the outcome, and no important omitted non-linearity.
- Independent errors, constant error variance and approximately normal residuals.
- No severe multicollinearity (check VIF and the predictor correlation matrix). O'Brien (2007) cautions against mechanical VIF cut-offs, so judge VIF alongside the width of the intervals.
- No influential outliers (leverage, Cook's distance).
Original example (illustrative)
Question. Do weekly study hours and class attendance independently predict exam score? Data for n = 10 students (invented):
| Student | Hours | Attendance (%) | Score |
|---|---|---|---|
| 1 | 2 | 60 | 55 |
| 2 | 3 | 75 | 60 |
| 3 | 4 | 70 | 62 |
| 4 | 5 | 80 | 68 |
| 5 | 6 | 85 | 72 |
| 6 | 7 | 78 | 70 |
| 7 | 8 | 90 | 80 |
| 8 | 9 | 88 | 78 |
| 9 | 10 | 95 | 88 |
| 10 | 12 | 92 | 90 |
| Term | b | SE | t | p | 95% CI |
|---|---|---|---|---|---|
| Intercept | 26.092 | 7.673 | 3.40 | .011 | [7.95, 44.24] |
| Study hours | 2.449 | 0.420 | 5.83 | < .001 | [1.46, 3.44] |
| Attendance (%) | 0.370 | 0.123 | 3.00 | .020 | [0.08, 0.66] |
Model fit: R² = 0.981, adjusted R² = 0.976, F(2, 7) = 185.40, p < .001. The correlation between the two predictors is r = 0.89, giving VIF = 1/(1 − 0.80) = 4.99 for each, which signals that collinearity is present and inflates the standard errors. Standardised coefficients are βhours = 0.67 and βattendance = 0.34.
Interpretation. The model explains most of the variance in scores. Holding attendance constant, each extra weekly hour is associated with about 2.45 more points. Holding hours constant, each extra percentage point of attendance is associated with about 0.37 points. Because the predictors are correlated, the individual intervals are wider than they would be for uncorrelated predictors, even though the overall model is clearly useful. With n = 10 and two predictors this is a teaching example; real analyses need larger samples.
How to run multiple linear regression with StatFAS
Open the Multiple Linear Regression Calculator. Arrange one numeric outcome column and one numeric column per predictor.
Understanding the output
- Coefficient table: unstandardised b for interpretation in original units, standardised β for relative comparison.
- R² and adjusted R²: adjusted R² penalises extra predictors.
- Overall F: tests whether at least one slope differs from zero.
- VIF/tolerance: collinearity diagnostics.
- Residual plots: check linearity, spread and outliers.
APA 7 reporting example
A multiple linear regression with study hours and attendance as predictors was significant, F(2, 7) = 185.40, p < .001, R² = .98, adjusted R² = .98. Study hours (b = 2.45, 95% CI [1.46, 3.44]) and attendance (b = 0.37, 95% CI [0.08, 0.66]) were estimated holding the other predictor constant.
Common mistakes and troubleshooting
- Too many predictors for the sample size, which overfits. Reduce predictors on theoretical grounds.
- Ignoring collinearity. Combine or drop redundant predictors, or centre variables when including interactions.
- Comparing raw coefficients across predictors with different units. Use standardised coefficients or interpret in context.
- Data-driven selection (stepwise) producing inflated, unstable results; see stepwise regression.
- Causal language after "controlling for" variables: adjustment does not replace design.
Limitations and alternatives
If a predictor's effect depends on another, test an interaction (see moderation analysis). If predictors enter in blocks for theoretical reasons, use hierarchical regression. For many correlated predictors, consider ridge or lasso regression. For one predictor, see simple linear regression.
Conclusion
Multiple regression lets you estimate adjusted associations, but only under reasonable assumptions, adequate sample size and careful attention to collinearity and design.
1.
statfas-multiple-linear-regression-interface.png, alt: "StatFAS Multiple Linear Regression Calculator interface"2.
statfas-multiple-linear-regression-example-data.png, alt: "Example data entered in the StatFAS Multiple Linear Regression Calculator"3.
statfas-multiple-linear-regression-settings.png, alt: "Multiple Linear Regression Calculator settings in StatFAS"4.
statfas-multiple-linear-regression-result.png, alt: "Multiple Linear Regression Calculator results generated in StatFAS"Frequently asked questions
How many predictors can I include?
It depends on sample size, effect size and the precision you need. Rules of thumb such as 10 to 20 cases per predictor are only rough guides; overfitting becomes likely when predictors approach the sample size.
What is multicollinearity and why does it matter?
It occurs when predictors are highly correlated. It does not bias the model's predictions but inflates standard errors, making individual coefficients unstable and hard to interpret.
Should I report standardised or unstandardised coefficients?
Report unstandardised coefficients with confidence intervals for interpretation in original units, and add standardised coefficients if you need to compare predictors measured on different scales.
References
- Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Erlbaum.
- O'Brien, R. M. (2007). A caution regarding rules of thumb for variance inflation factors. Quality & Quantity, 41(5), 673–690.