Feature Importance Plot: How to Rank Genes, Biomarkers and Variables in a Model
A feature importance plot (also called a variable importance plot) ranks the predictors of a model by how much the model depends on each predictor. This guide explains what the feature importance plot shows, when and why to use the feature importance plot, the key points of interpretation, and how to create the feature importance plot in SPSS, R and Python or in seconds on statfas.com.
What is a feature importance plot?
A feature importance plot (also called a variable importance plot) ranks the predictors of a model by how much the model depends on each predictor. Bars show the importance of each feature, often with error bars for uncertainty.
Importance can come from impurity (Gini) in tree models, from coefficients in linear models, or from permutation importance: shuffle one feature and measure how much performance drops. Permutation importance works with any model.
When and why should you use a feature importance plot?
Use a feature importance plot in these situations:
- Biomarker and gene discovery: deciding which genes, proteins or clinical variables drive a prediction.
- Model explanation: showing clinicians or reviewers what a machine-learning model uses.
- Feature selection: dropping variables that add only noise before refitting a simpler model.
- Model auditing: catching leakage, for example an ID or a post-outcome variable ranked first.
A feature importance plot turns a black-box model into a ranked list that can be checked against biology and clinical knowledge, and the feature importance plot is increasingly expected in machine-learning papers.
How to interpret a feature importance plot: key points
- Name the importance measure. Permutation importance on held-out data is generally preferred, because impurity importance can favour variables with many distinct values (Altmann et al., 2010).
- Use repeats for uncertainty. With the mean, standard deviation and number of repeats, a t-based confidence interval and Holm-adjusted p value can be reported. These p values describe stability across shuffles for this model and dataset, not causal effects.
- Correlated features share importance. A truly relevant gene can look unimportant when a correlated gene absorbs the importance. Group correlated features.
- Importance is model-specific. Another algorithm can rank the same variables differently (Fisher et al., 2019).
- Look at relative and cumulative importance to decide how many features matter, but remember that shares depend on which features were included.
- Importance is not direction. A feature importance plot does not show whether a variable raises or lowers risk.
How to make a feature importance plot in SPSS, R and Python
The snippets below show the core commands; the full feature importance plot tutorial has complete, copy-and-run code. Examples assume a data frame df with the columns named in the code.
Feature Importance Plot in SPSS
SPSS Statistics has no permutation importance menu. Two built-in options give a similar ranking: Analyze > Regression > Automatic Linear Modeling (predictor importance chart for a continuous outcome) and standardised coefficients from Analyze > Regression > Binary Logistic or Linear after z-scoring the predictors. For permutation importance with a random forest, run the R or Python code below (SPSS can call both), then plot the table.
SPSS syntax
* Standardise predictors, then compare standardised coefficients.
DESCRIPTIVES VARIABLES=x1 x2 x3 /SAVE.
LOGISTIC REGRESSION VARIABLES outcome
/METHOD=ENTER Zx1 Zx2 Zx3
/PRINT=CI(95).Feature Importance Plot in R
R code
library(ranger)
rf <- ranger(outcome ~ ., data = train, importance = "permutation")
sort(importance(rf), decreasing = TRUE)Feature Importance Plot in Python
Python code
from sklearn.inspection import permutation_importance
r = permutation_importance(rf, X_test, y_test, scoring="roc_auc", n_repeats=30, random_state=1)
r.importances_mean, r.importances_stdThe easy way: free feature importance plot maker on statfas.com
The Feature Importance Plot maker on statfas.com uploads an Excel, CSV or SPSS table with feature, importance, SD and repeats, ranks the features, draws the feature importance plot with 95% confidence intervals, and returns relative and cumulative importance, t tests and Holm-adjusted p values. No coding is needed: upload the file, confirm the columns and read the results.
- Edit the feature importance plot: title, axis labels, colours, line and marker sizes, fonts, grid lines and resolution (150, 300 or 600 dpi).
- Copy ready-to-use APA 7 text for free.
- Download the feature importance plot (PNG or SVG), the tables (CSV) and a Word report that combines the table, the figure and the interpretation after a free sign-up and log-in.
Try the feature importance plot maker
How to report a feature importance plot in APA 7
Example wording (illustrative values; replace with your own results):
Place the figure number and title above the feature importance plot and a note below, as the APA 7 style requires. The statfas.com feature importance plot maker writes the caption and results paragraph automatically.
Frequently asked questions
- What is permutation importance?
- Permutation importance measures the drop in model performance when one feature is shuffled. A large drop means the model relies on that feature.
- Is random forest Gini importance reliable?
- Gini importance can favour continuous or high-cardinality variables. Permutation importance on held-out data is usually more trustworthy.
- Does a high importance mean a causal effect?
- No. Feature importance shows how much a model uses a variable, not whether the variable causes the outcome or in which direction the variable acts.
- Why do correlated genes look unimportant?
- Correlated features share importance, so shuffling one feature loses little because a correlated feature carries the same information. Group correlated features or report them together.
References
Altmann, A., Toloşi, L., Sander, O., & Lengauer, T. (2010). Permutation importance: A corrected feature importance measure. Bioinformatics, 26(10), 1340–1347.
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.
Fisher, A., Rudin, C., & Dominici, F. (2019). All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously. Journal of Machine Learning Research, 20(177), 1–81.