Lasso Regression Explained: How to Run, Read and Report It

By Dr. Faheem Afzal Shah, Founder, StatFAS.com · Tool: Lasso Regression Calculator

Lasso (least absolute shrinkage and selection operator) regression shrinks coefficients like ridge regression, but its penalty can set some coefficients exactly to zero. It therefore performs estimation and variable selection in one step, which makes it popular when you have many candidate predictors and expect only some to matter.

What is lasso regression?

Lasso adds an L1 penalty to the least-squares criterion (Tibshirani, 1996):

β̂lasso = argmin [ (1 / 2n) Σ(yi − xi′β)² + λ Σ|βj| ]

(The 1/2n scaling is that of the scikit-learn and glmnet convention; other software scales λ differently, so λ values are not comparable across programs.) As λ increases, more coefficients are driven exactly to zero. Predictors are standardised first so that the penalty is applied fairly, and the intercept is not penalised. λ is chosen by cross-validation (Friedman et al., 2010).

When should you use it?

Use it when you have many predictors relative to observations, want a sparse, interpretable model, or need to reduce overfitting. Compare it with ridge regression, which keeps all predictors, and with ordinary multiple regression when predictors are few and well understood.

Assumptions and caveats

Original example (illustrative)

Question. Which of eight candidate measures (x₁ to x₈) predict a performance score? Data for n = 40 (invented). In this simulated dataset I set only x₁, x₂ and x₃ to have real effects, with x₂ moderately correlated with x₁, and the others have none; real data come without that knowledge.

Show the full illustrative dataset (CSV, copy into the tool)
x1,x2,x3,x4,x5,x6,x7,x8,score
51.3,49.5,56.4,51.0,44.6,53.6,63.0,59.5,65.8
43.0,35.5,43.8,50.4,26.7,47.8,37.5,42.7,38.8
44.6,44.5,54.1,60.4,48.7,63.7,43.3,53.5,47.1
59.0,55.3,42.6,40.8,45.4,52.2,39.9,47.9,69.8
48.4,53.9,52.1,53.6,43.5,48.7,57.8,64.9,67.6
37.4,56.9,63.5,57.8,52.6,46.9,64.6,69.6,58.2
68.0,70.4,53.6,37.9,50.0,56.6,37.1,54.0,92.6
54.3,58.2,38.2,43.4,45.6,38.3,67.4,45.0,61.2
53.3,49.4,65.8,63.2,56.3,28.0,50.5,56.8,70.1
60.0,49.6,68.2,36.8,43.4,59.4,50.5,70.0,75.3
51.9,45.4,46.2,39.1,37.2,56.3,55.8,62.9,59.0
42.5,60.9,47.1,65.7,45.7,42.6,52.5,60.3,49.4
51.6,45.7,36.6,36.0,55.0,59.9,48.4,39.3,56.7
58.7,43.2,42.9,56.2,27.5,53.9,44.2,51.1,59.2
49.2,51.4,56.9,42.4,64.2,57.3,58.4,61.6,65.4
57.9,61.3,50.8,35.7,48.6,42.3,35.8,52.6,72.2
44.3,38.2,39.6,52.7,53.6,63.2,49.9,60.4,45.0
64.0,67.0,26.3,62.3,53.4,54.2,53.7,53.8,65.8
53.2,48.5,31.0,48.9,42.0,60.8,47.1,50.8,60.5
41.5,41.3,49.9,35.1,53.0,48.9,38.1,26.0,36.9
55.1,50.0,44.7,47.6,68.2,49.5,50.9,35.1,56.0
66.5,66.2,60.7,50.5,59.2,53.7,56.1,48.5,90.8
35.3,51.6,30.7,47.6,48.0,39.6,56.1,48.0,47.8
45.6,52.3,45.2,63.9,53.5,45.3,30.6,36.9,57.0
60.9,55.0,47.2,66.4,37.2,44.1,45.3,55.9,70.0
43.4,41.3,33.9,57.3,58.1,45.2,51.6,37.1,33.1
45.3,59.6,51.4,73.1,42.1,55.8,48.0,55.7,61.0
49.9,45.1,41.3,80.7,49.2,29.8,43.5,56.8,53.0
45.0,59.3,60.0,48.5,45.3,40.0,43.0,35.3,75.8
62.0,69.9,37.4,38.2,32.3,40.4,18.9,38.6,80.8
63.0,53.5,58.5,45.1,67.6,52.0,46.2,75.5,68.4
46.8,37.8,52.0,49.6,60.7,40.8,58.0,58.5,62.8
43.3,48.1,41.7,73.5,43.0,45.5,39.3,46.5,46.7
49.9,56.7,43.9,48.1,35.8,41.7,77.6,60.4,55.9
42.2,34.5,40.2,49.8,50.3,42.6,37.1,64.2,48.1
54.5,49.0,47.8,44.7,20.6,51.2,39.3,40.0,62.2
43.6,53.2,38.3,35.7,56.4,57.5,40.4,55.6,48.7
47.1,51.2,37.4,58.3,62.0,56.4,55.6,12.3,53.7
52.6,51.1,48.5,43.7,50.6,54.1,47.4,45.4,67.4
62.3,46.5,60.3,51.8,42.0,47.1,40.8,56.8,79.6

Five-fold cross-validation selects λ = 0.774 (scikit-learn scale). The table shows standardised coefficients for OLS and for lasso at this λ.

PredictorOLSLasso
x17.086.86
x24.854.32
x34.594.04
x4−1.13−0.21
x5−0.350 (dropped)
x6−0.770 (dropped)
x7−0.270 (dropped)
x80.340 (dropped)
Number of non-zero coefficients along the penalty path
λNon-zero
6.1892
1.5473
0.774 (CV choice)4
0.1938

Lasso keeps 4 predictors (x1, x2, x3, x4). Cross-validated RMSE is 6.11 for lasso and 7.23 for OLS with all eight predictors; the in-sample R² of the lasso fit is 0.83 (in-sample, so optimistic).

Interpretation. The lasso keeps the three predictors that were given real effects (x₁ to x₃) and also retains x4, which has no true effect (a false positive that selection methods can produce), and drops the other 4 predictors, with coefficients shrunk toward zero compared with OLS. The shrinkage is deliberate bias that buys lower variance. Retention of a variable means it helped predict held-out data under this penalty, not that it has a proven effect, and the selected set would likely change somewhat in a new sample. Treat the output as a data-driven shortlist to be checked in new data.

How to run lasso regression with StatFAS

Open the Lasso Regression Calculator. Arrange one outcome column and several predictor columns.

Understanding the output

APA 7 reporting example

Lasso regression with standardised predictors and λ = 0.774 (five-fold cross-validation; scikit-learn scale) retained 4 of 8 predictors (x1, x2, x3, x4). Cross-validated RMSE was 6.11 for lasso and 7.23 for the full OLS model.

Common mistakes and troubleshooting

Limitations and alternatives

With groups of correlated predictors, elastic net is usually preferable. If you do not need selection, ridge regression often predicts as well. For theory-driven entry of predictors use hierarchical regression; stepwise methods are generally less reliable than penalised regression (stepwise regression). See also the regression overview.

Conclusion

Lasso offers selection and shrinkage together. Standardise, tune λ by cross-validation, evaluate on held-out data, and treat the selected variables as hypotheses rather than established effects.

Screenshots to supply (genuine captures only):
1. statfas-lasso-regression-interface.png, alt: "StatFAS Lasso Regression Calculator interface"
2. statfas-lasso-regression-example-data.png, alt: "Example data entered in the StatFAS Lasso Regression Calculator"
3. statfas-lasso-regression-settings.png, alt: "Lasso Regression Calculator settings in StatFAS"
4. statfas-lasso-regression-result.png, alt: "Lasso Regression Calculator results generated in StatFAS"

Frequently asked questions

Does lasso regression give p-values?

Not valid ones in the usual sense. Standard errors and p-values do not account for the selection step. Use post-selection inference methods, bootstrap stability checks or validate the selected model on new data.

When should I use lasso instead of ridge?

Use lasso when you expect only a subset of predictors to matter and want a sparse model. Use ridge when many predictors each contribute a little or when predictors are strongly correlated and you want to keep all of them.

What is the one-standard-error rule?

It chooses the largest λ whose cross-validated error is within one standard error of the minimum, giving a simpler model with similar predictive performance.

References

Author attribution: Dr. Faheem Afzal Shah, Plant Functional Genomics and Regulatory Genomics Researcher; StatFAS.com Founder.