ROC Curve and AUC: What They Mean, When to Use Them and How to Interpret Them
A ROC curve (receiver operating characteristic curve) is a graph that shows how well a test, biomarker or machine-learning model separates two groups, such as patients with and without a disease. This guide explains what the ROC curve shows, when and why to use the ROC curve, the key points of interpretation, and how to create the ROC curve in SPSS, R and Python or in seconds on statfas.com.
What is a ROC curve?
A ROC curve (receiver operating characteristic curve) is a graph that shows how well a test, biomarker or machine-learning model separates two groups, such as patients with and without a disease. The ROC curve plots sensitivity (the true-positive rate) on the y-axis against 1 − specificity (the false-positive rate) on the x-axis for every possible cut-off value.
The area under the ROC curve (AUC) condenses the whole ROC curve into one number between 0 and 1. An AUC of .50 means no better than a coin flip; an AUC of 1.00 means perfect separation. The AUC also equals the probability that a randomly chosen positive case scores higher than a randomly chosen negative case.
When and why should you use a ROC curve?
Use a ROC curve in these situations:
- Diagnostic accuracy studies: judging a serum marker, imaging score or gene-expression signature against a reference standard.
- Machine-learning classifiers: evaluating logistic regression, random forest or neural-network predictions on clinical, genomic or proteomic data.
- Choosing a cut-off: turning a continuous measurement into positive or negative when the ROC curve gives sensitivity and specificity at every threshold.
- Comparing markers: ranking candidate biomarkers by discrimination when the groups are reasonably balanced.
Researchers use a ROC curve because the ROC curve does not depend on one arbitrary threshold. Journals and reviewers also expect an AUC with a confidence interval and a p value against chance.
How to interpret a ROC curve: key points
- AUC = .50 is chance. A common rule of thumb calls .70–.80 acceptable, .80–.90 excellent and .90 or higher outstanding (Hosmer & Lemeshow, 2000), but clinical needs decide what is good enough.
- Read the 95% confidence interval. The lower limit should stay clearly above .50; a wide interval signals a small sample.
- Shape matters. A ROC curve hugging the top-left corner is best. A ROC curve below the diagonal means the score direction is reversed.
- Pick a cut-off deliberately. The Youden index (sensitivity + specificity − 1) picks the point farthest above the diagonal; report sensitivity and specificity with confidence intervals at that point.
- The ROC curve ignores calibration and prevalence. Add a calibration plot, and use a precision–recall curve when positives are rare (Saito & Rehmsmeier, 2015).
- Use held-out data. A ROC curve drawn on the training set overstates real performance.
How to make a ROC curve in SPSS, R and Python
The snippets below show the core commands; the full ROC curve tutorial has complete, copy-and-run code. Examples assume a data frame df with the columns named in the code.
ROC Curve in SPSS
Choose Analyze > Classification > ROC Curve. Move the score to Test Variable, the outcome to State Variable and type the value that means positive (for example 1) in Value of State Variable. Under Display tick ROC Curve, With diagonal reference line, Standard error and confidence interval, and Coordinate points of the ROC Curve. Click OK.
SPSS syntax
ROC score BY outcome (1)
/PLOT=CURVE(REFERENCE)
/PRINT=SE COORDINATES
/CRITERIA=CUTOFF(INCLUDE) TESTPOS(LARGE) DISTRIBUTION(FREE) CI(95)
/MISSING=EXCLUDE.ROC Curve in R
R code
library(pROC)
roc_obj <- roc(df$outcome, df$score, levels = c(0, 1), direction = "<")
auc(roc_obj); ci.auc(roc_obj)ROC Curve in Python
Python code
from sklearn.metrics import roc_auc_score, roc_curve
auc = roc_auc_score(df["outcome"], df["score"])
fpr, tpr, thr = roc_curve(df["outcome"], df["score"])The easy way: free ROC curve maker on statfas.com
The ROC Curve maker on statfas.com uploads an Excel, CSV or SPSS file, draws the ROC curve, and returns the AUC with Hanley–McNeil standard error, 95% CI, a z-test against .50, and the Youden cut-off with sensitivity and specificity. No coding is needed: upload the file, confirm the columns and read the results.
- Edit the ROC curve: title, axis labels, colours, line and marker sizes, fonts, grid lines and resolution (150, 300 or 600 dpi).
- Copy ready-to-use APA 7 text for free.
- Download the ROC curve (PNG or SVG), the tables (CSV) and a Word report that combines the table, the figure and the interpretation after a free sign-up and log-in.
How to report a ROC curve in APA 7
Example wording (illustrative values; replace with your own results):
Place the figure number and title above the ROC curve and a note below, as the APA 7 style requires. The statfas.com ROC curve maker writes the caption and results paragraph automatically.
Frequently asked questions
- What is a good AUC value?
- An AUC of .70–.80 is usually called acceptable, .80–.90 excellent and above .90 outstanding. Always judge the AUC together with the 95% confidence interval and the clinical consequences of errors.
- What does an AUC of 0.5 mean?
- An AUC of .50 means the score cannot separate the two groups and performs like random guessing. An AUC below .50 usually means the score direction is reversed.
- How do I choose the best cut-off on a ROC curve?
- Many studies use the Youden index (sensitivity + specificity − 1). If missing a case is costlier than a false alarm, choose a cut-off that favours sensitivity and explain the choice.
- Can a ROC curve be used with imbalanced data?
- A ROC curve can be computed, but the ROC curve may look good even when many false positives occur. A precision–recall curve is more informative when positives are rare.
References
Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1), 29–36.
Hosmer, D. W., & Lemeshow, S. (2000). Applied logistic regression (2nd ed.). Wiley.
Saito, T., & Rehmsmeier, M. (2015). The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLOS ONE, 10(3), Article e0118432.
Youden, W. J. (1950). Index for rating diagnostic tests. Cancer, 3(1), 32–35.