Calibration Plot (Calibration Curve): Do Predicted Probabilities Match Observed Risk?
A calibration plot (also called a calibration curve or reliability diagram) compares predicted probabilities with observed outcomes. This guide explains what the calibration plot shows, when and why to use the calibration plot, the key points of interpretation, and how to create the calibration plot in SPSS, R and Python or in seconds on statfas.com.
What is a calibration plot?
A calibration plot (also called a calibration curve or reliability diagram) compares predicted probabilities with observed outcomes. Cases are sorted by predicted risk and grouped; for each group the calibration plot shows the mean predicted probability on the x-axis and the observed event proportion on the y-axis.
Points on the diagonal mean perfect calibration. Points below the diagonal mean the model overestimates risk; points above mean the model underestimates risk.
When and why should you use a calibration plot?
Use a calibration plot in these situations:
- Clinical prediction models: any model that outputs a risk, such as 10-year cardiovascular risk or recurrence probability.
- Before using probabilities for decisions: treatment thresholds and risk communication need numbers that mean what the numbers say.
- Validating a model on new data: calibration usually degrades first (Van Calster et al., 2019).
- Alongside discrimination: a ROC curve cannot detect miscalibration, so the calibration plot completes the evaluation.
A model with an excellent AUC can still say “30% risk” for patients whose true risk is 10%. The calibration plot is the standard check for that failure and the TRIPOD reporting guideline for prediction models recommends reporting calibration.
How to interpret a calibration plot: key points
- Calibration slope should be 1. A slope below 1 means predictions are too extreme (overfitting); above 1 means predictions are too moderate.
- Calibration intercept should be 0. A negative intercept means the model overestimates risk overall; a positive intercept means the model underestimates risk.
- Brier score is the mean squared error of the probabilities (lower is better); compare the Brier score with a prevalence-only model using the skill score.
- Hosmer–Lemeshow test is secondary. The test depends on the number of groups and loses power in small samples, so rely on the calibration plot, slope and intercept first.
- Look at confidence intervals around the points. Groups with few cases are noisy, especially at high predicted risk.
- Use new data. A calibration plot on the training data hides overfitting.
How to make a calibration plot in SPSS, R and Python
The snippets below show the core commands; the full calibration plot tutorial has complete, copy-and-run code. Examples assume a data frame df with the columns named in the code.
Calibration Plot in SPSS
SPSS has no calibration plot menu, but all pieces are available. (1) Hosmer–Lemeshow: Analyze > Regression > Binary Logistic, enter your predictors, click Options and tick Hosmer-Lemeshow goodness-of-fit. (2) Calibration slope and intercept: compute the log-odds of the predicted probability and regress the outcome on the log-odds. (3) Calibration points: group the predicted probability into deciles, aggregate, and plot observed against predicted.
SPSS syntax
COMPUTE lp = LN(prob/(1-prob)).
EXECUTE.
* Calibration slope = B for lp; calibration intercept = Constant.
LOGISTIC REGRESSION VARIABLES outcome
/METHOD=ENTER lp
/PRINT=CI(95).
* Calibration points: deciles of predicted risk.
RANK VARIABLES=prob (A) /NTILES(10) INTO decile.
AGGREGATE /OUTFILE=* /BREAK=decile
/pred_mean=MEAN(prob) /obs_rate=MEAN(outcome) /n=N.
GRAPH /SCATTERPLOT(BIVAR)=pred_mean WITH obs_rate.Calibration Plot in R
R code
library(rms)
val.prob(df$prob, df$outcome) # slope, intercept, Brier score and plotCalibration Plot in Python
Python code
from sklearn.calibration import calibration_curve
obs, pred = calibration_curve(df["outcome"], df["prob"], n_bins=10, strategy="quantile")The easy way: free calibration plot maker on statfas.com
The Calibration Plot maker on statfas.com uploads an Excel, CSV or SPSS file with the outcome and the predicted probability, draws the calibration plot with Wilson 95% intervals, and returns the calibration slope and intercept with tests, Hosmer–Lemeshow, the Brier score and skill, and the expected calibration error. No coding is needed: upload the file, confirm the columns and read the results.
- Edit the calibration plot: title, axis labels, colours, line and marker sizes, fonts, grid lines and resolution (150, 300 or 600 dpi).
- Copy ready-to-use APA 7 text for free.
- Download the calibration plot (PNG or SVG), the tables (CSV) and a Word report that combines the table, the figure and the interpretation after a free sign-up and log-in.
Try the calibration plot maker
How to report a calibration plot in APA 7
Example wording (illustrative values; replace with your own results):
Place the figure number and title above the calibration plot and a note below, as the APA 7 style requires. The statfas.com calibration plot maker writes the caption and results paragraph automatically.
Frequently asked questions
- What is a calibration plot?
- A calibration plot compares predicted probabilities with observed event rates. Points on the diagonal indicate perfect calibration.
- What is a good calibration slope?
- A calibration slope close to 1 is ideal. Values below 1 point to overfitting, and values above 1 mean predictions are too moderate.
- Is the Hosmer-Lemeshow test enough to prove calibration?
- No. The Hosmer–Lemeshow test depends on grouping and sample size. Report the calibration plot, slope, intercept and Brier score as well.
- What is the difference between calibration and discrimination?
- Discrimination (AUC) is the ability to rank patients correctly. Calibration is the agreement between predicted and observed risk. A model needs both.
References
Hosmer, D. W., & Lemeshow, S. (2000). Applied logistic regression (2nd ed.). Wiley.
Van Calster, B., McLernon, D. J., van Smeden, M., Wynants, L., & Steyerberg, E. W. (2019). Calibration: The Achilles heel of predictive analytics. BMC Medicine, 17, Article 230.