Simple Linear Regression Explained: How to Run, Read and Report It
Linear regression estimates how an outcome changes, on average, when a predictor changes by one unit. Simple linear regression uses one continuous predictor and one continuous outcome, and it gives you an equation, a measure of fit and an interval for the slope. This guide explains the method with a fully worked example.
What is simple linear regression?
It fits a straight line, ŷ = b0 + b1x, by ordinary least squares (OLS): the line that minimises the sum of squared vertical distances between observed and predicted values. The slope b1 is the expected change in y per one-unit increase in x. The intercept b0 is the expected y when x = 0, which may not be meaningful if x = 0 lies outside the observed range.
When should you use it?
Use it to describe or predict a continuous outcome from one continuous predictor: exam score from study hours, yield from fertiliser dose, reaction time from age. It is not appropriate for a binary outcome (see logistic regression) or for strongly clustered or repeated observations without modification. Correlation and regression are related, but regression is directional and gives a prediction equation.
Mathematical foundation
b1 = Sxy / Sxx b0 = ȳ − b1x̄ R² = SSreg / SStotal SE(b1) = s / √Sxx
Here Sxx = Σ(x − x̄)², Sxy = Σ(x − x̄)(y − ȳ), and s = √MSE is the residual standard error with n − 2 degrees of freedom. The test of the slope is t = b1/SE(b1) with df = n − 2, and in simple regression F = t².
Assumptions
- Linearity of the relationship (check a residual-versus-fitted plot).
- Independence of observations.
- Homoscedasticity: residual spread is roughly constant across fitted values.
- Normally distributed residuals (matters mainly for small samples; check a Q-Q plot).
- No unduly influential points (check leverage and Cook's distance). The predictor is assumed measured without substantial error.
Original example (illustrative)
Background and question. A lecturer asks whether weekly study hours predict exam score. H0: β1 = 0; H1: β1 ≠ 0. Data for n = 8 students (invented for teaching):
| Student | Hours (x) | Score (y) | Fitted ŷ | Residual |
|---|---|---|---|---|
| 1 | 1 | 52 | 52.11 | −0.11 |
| 2 | 2 | 55 | 56.39 | −1.39 |
| 3 | 2 | 58 | 56.39 | 1.61 |
| 4 | 3 | 60 | 60.67 | −0.67 |
| 5 | 4 | 66 | 64.94 | 1.06 |
| 6 | 5 | 68 | 69.22 | −1.22 |
| 7 | 6 | 75 | 73.50 | 1.50 |
| 8 | 7 | 77 | 77.78 | −0.78 |
x̄ = 3.75, ȳ = 63.88. Sxx = 31.50, Sxy = 134.75, Syy = 586.88. Therefore b1 = 134.75/31.50 = 4.278 and b0 = 63.88 − 4.278 × 3.75 = 47.83. The fitted line is ŷ = 47.83 + 4.28x. SSreg = Sxy²/Sxx = 576.43, SSres = 10.44, so R² = 0.982. With df = 6, MSE = 1.741, s = 1.32 and SE(b1) = 0.235. Then t(6) = 18.20, p < .001, equivalently F(1, 6) = 331.14. The 95% CI for the slope is 4.278 ± 2.447 × 0.235 = [3.70, 4.85]. (The residuals sum to zero, which is a useful arithmetic check.)
Interpretation. Each additional weekly study hour is associated with an average score increase of about 4.3 points (95% CI [3.7, 4.9]), and study hours account for 98.2% of the variance in these eight scores. The very high R² is a feature of these invented numbers; real behavioural data usually show much lower values. The result describes association, not causation, and predictions should stay within the observed range of 1 to 7 hours.
How to run linear regression with StatFAS
Open the Linear Regression Calculator. Arrange the data as two numeric columns, predictor and outcome, with one row per participant. Check the column order, because swapping x and y gives a different equation.
Understanding the output
- Slope and intercept with standard errors, t and p: the slope is the key estimate; interpret it in units.
- Confidence interval for the slope: shows the plausible range of the effect.
- R² and adjusted R²: proportion of variance explained; a high R² does not prove the model is correct.
- F-test: overall significance; identical in meaning to the slope test here.
- Residual plots: look for curvature, funnels and outliers.
APA 7 reporting example
A simple linear regression showed that weekly study hours significantly predicted exam score, F(1, 6) = 331.14, p < .001, R² = .98. Each additional hour was associated with a 4.28-point increase in score (b = 4.28, 95% CI [3.70, 4.85]).
Common mistakes and troubleshooting
- Extrapolating beyond the data range. A line fitted to 1–7 hours says nothing reliable about 20 hours.
- Reading causation from observational data.
- Ignoring outliers or influential points. A single extreme x-value can drive the slope; refit without it as a sensitivity check and report both.
- Using regression on curved relationships. If residuals curve, add a quadratic term (see polynomial regression) or transform.
- Reporting only p. Always report the slope with its interval.
Limitations and alternatives
With one predictor, omitted variables can bias the slope; consider multiple linear regression. For binary outcomes use logistic regression. For non-linear patterns use polynomial regression. If you only need the strength of association, the correlation calculator may be enough.
Conclusion
Simple linear regression turns an association into an estimable effect with uncertainty. Report the slope, its confidence interval, the fit and the diagnostics, and treat the result as description unless the design supports causal claims.
1.
statfas-linear-regression-interface.png, alt: "StatFAS Linear Regression Calculator interface"2.
statfas-linear-regression-example-data.png, alt: "Example data entered in the StatFAS Linear Regression Calculator"3.
statfas-linear-regression-settings.png, alt: "Linear Regression Calculator settings in StatFAS"4.
statfas-linear-regression-result.png, alt: "Linear Regression Calculator results generated in StatFAS"Frequently asked questions
What is a good R² value?
There is no universal threshold. Acceptable values depend on the field; in many behavioural studies an R² of .10 to .30 is common. Judge the model by its interval estimates, diagnostics and purpose, not only R².
What is the difference between correlation and regression?
Correlation measures the strength and direction of a linear association symmetrically. Regression models an outcome from a predictor, giving a prediction equation in the outcome's units.
Can I use linear regression with a small sample?
Yes, but estimates are imprecise and assumption checks are harder. Report confidence intervals and be cautious about generalising.
References
- Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Erlbaum.
- Draper, N. R., & Smith, H. (1998). Applied regression analysis (3rd ed.). Wiley.