Confusion Matrix Explained: Accuracy, Sensitivity, Specificity, F1, MCC and Kappa
A confusion matrix is a table that compares the predicted class with the actual class for every case at one chosen threshold. This guide explains what the confusion matrix shows, when and why to use the confusion matrix, the key points of interpretation, and how to create the confusion matrix in SPSS, R and Python or in seconds on statfas.com.
What is a confusion matrix?
A confusion matrix is a table that compares the predicted class with the actual class for every case at one chosen threshold. For a yes/no outcome the confusion matrix has four cells: true positives (TP), false positives (FP), false negatives (FN) and true negatives (TN).
Almost every classification metric comes from those four cells: accuracy, sensitivity (recall), specificity, positive and negative predictive value, F1, the Matthews correlation coefficient (MCC) and Cohen’s kappa.
When and why should you use a confusion matrix?
Use a confusion matrix in these situations:
- Reporting a classifier at a fixed cut-off: a diagnostic test with a clinical threshold, or a model deployed at 0.5.
- Seeing how errors split: the confusion matrix shows whether the model mostly misses cases (false negatives) or raises false alarms (false positives).
- Reporting several metrics at once: sensitivity, specificity, PPV and NPV all come from the confusion matrix.
- Checking agreement: Cohen’s kappa and the MCC summarise how far predictions beat chance.
The confusion matrix is the most transparent view of classification errors, and reviewers often ask for the raw counts behind every percentage.
How to interpret a confusion matrix: key points
- Rows are actual classes, columns are predicted classes in most software, but some packages swap the axes. Check the labels before interpreting the confusion matrix.
- Accuracy can mislead. If 95% of patients are healthy, predicting “healthy” for everyone gives 95% accuracy. Compare accuracy with the no-information rate.
- Sensitivity and specificity do not depend on prevalence; PPV and NPV do. A positive result in a rare disease is often a false alarm.
- Prefer balanced summaries. The MCC and kappa use all four cells and remain informative on imbalanced data (Chicco & Jurman, 2020; Landis & Koch, 1977).
- One confusion matrix belongs to one threshold. State the threshold, or justify the threshold with a ROC analysis on separate data.
- Show row percentages. Percentages within each actual class make recall visible even with unequal class sizes.
How to make a confusion matrix in SPSS, R and Python
The snippets below show the core commands; the full confusion matrix tutorial has complete, copy-and-run code. Examples assume a data frame df with the columns named in the code.
Confusion Matrix in SPSS
First create the predicted class: Transform > Compute Variable, target pred, expression score >= 0.5. Then choose Analyze > Descriptive Statistics > Crosstabs. Put the actual outcome in Row(s) and pred in Column(s). Under Statistics tick Chi-square, Phi and Cramér’s V, and Kappa. Under Cells tick Observed and Row percentages.
SPSS syntax
COMPUTE pred = (score >= 0.5).
EXECUTE.
CROSSTABS
/TABLES=outcome BY pred
/STATISTICS=CHISQ PHI KAPPA
/CELLS=COUNT ROW.Confusion Matrix in R
R code
library(caret)
confusionMatrix(factor(df$score >= 0.5, levels = c(FALSE, TRUE), labels = c(0, 1)),
factor(df$outcome, levels = c(0, 1)), positive = "1")Confusion Matrix in Python
Python code
from sklearn.metrics import confusion_matrix
pred = (df["score"] >= 0.5).astype(int)
cm = confusion_matrix(df["outcome"], pred) # [[TN, FP], [FN, TP]]The easy way: free confusion matrix maker on statfas.com
The Confusion Matrix maker on statfas.com uploads an Excel, CSV or SPSS file, lets you set the threshold, draws the confusion matrix with row percentages, and returns accuracy, sensitivity, specificity, PPV, NPV, F1, MCC, kappa, a chi-square test and the accuracy versus no-information-rate test, all with 95% confidence intervals. No coding is needed: upload the file, confirm the columns and read the results.
- Edit the confusion matrix: title, axis labels, colours, line and marker sizes, fonts, grid lines and resolution (150, 300 or 600 dpi).
- Copy ready-to-use APA 7 text for free.
- Download the confusion matrix (PNG or SVG), the tables (CSV) and a Word report that combines the table, the figure and the interpretation after a free sign-up and log-in.
Try the confusion matrix maker
How to report a confusion matrix in APA 7
Example wording (illustrative values; replace with your own results):
Place the figure number and title above the confusion matrix and a note below, as the APA 7 style requires. The statfas.com confusion matrix maker writes the caption and results paragraph automatically.
Frequently asked questions
- How do I read a confusion matrix?
- Rows show actual classes and columns show predicted classes in most software. The diagonal holds correct predictions; the off-diagonal cells hold false positives and false negatives.
- What is the difference between sensitivity and precision?
- Sensitivity (recall) is TP/(TP+FN), the share of real positives found. Precision (PPV) is TP/(TP+FP), the share of positive calls that are correct.
- Why is accuracy not enough?
- On imbalanced data a model can reach high accuracy by always predicting the majority class. Report sensitivity, specificity, MCC or kappa as well.
- What is a good kappa or MCC?
- Landis and Koch (1977) labelled kappa .41–.60 moderate, .61–.80 substantial and above .80 almost perfect. The MCC ranges from −1 to 1 and 0 means no better than chance.
References
Chicco, D., & Jurman, G. (2020). The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics, 21, Article 6.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.