Stop 07 · Model Evaluation & TuningTopics · 2/8

Confusion Matrix

In short

One accuracy number hides what kind of mistakes your model makes. A confusion matrix lays every right and wrong answer out in a small table you can read in seconds.

A table breaking predictions into true positives, false positives, true negatives and false negatives — the raw material every classification metric above is calculated from.

The mascot sorts ripe and raw mangoes into four baskets arranged in a square on a fruit stall.

Think of it like this

Sorting mangoes into four baskets

A fruit seller guesses ripe or raw for each mango, then cuts it open. Four baskets fill up: called ripe and was ripe, called ripe but was raw, called raw but was ripe, called raw and was raw. Two baskets are wins, two are different kinds of mistakes.

Watch it happen

Every prediction lands in one of four boxes

All predictions

Actually spam

Actually ham

TP: caught

FN: missed

FP: false alarm

TN: left alone

1/5Start with every email the model judged. First split by the real answer.

The big ideas

What you need to know

Rows are actual, columns predicted

That's the scikit-learn layout. Some tools flip it, so always check the axis labels before reading.

Diagonal means correct

Big numbers on the diagonal and small numbers elsewhere is what a good model looks like.

Every metric comes from it

Accuracy, precision, recall and F1 are all simple ratios of these four cells.

Works for many classes

With 5 classes you get a 5 by 5 grid. Off-diagonal cells show which classes get mixed up.

See it in code

Walk through the code

python

1from sklearn.metrics import confusion_matrix2 3y_true = ["spam", "ham", "spam", "ham", "ham", "spam", "ham", "ham"]4y_pred = ["spam", "ham", "ham", "ham", "spam", "spam", "ham", "ham"]5 6cm = confusion_matrix(y_true, y_pred, labels=["spam", "ham"])7print(cm)8# [[2 1]    actual spam: 2 caught, 1 missed9#  [1 4]]   actual ham: 1 false alarm, 4 correct

1/3Eight emails: the real label and the model's guess.

Passing labels fixes the row and column order, so you always know which cell is which.

Save yourself the pain

Common mistakes

Oops

Reading the matrix without checking which axis is actual and which is predicted.

Do this instead

Label your axes, or use ConfusionMatrixDisplay, which prints 'True label' and 'Predicted label'.

Oops

Judging a multi-class model only by its overall accuracy.

Do this instead

Look at off-diagonal cells to find which pairs of classes the model confuses, then fix those.

Your mission

Draw your own matrix

~20 min

Quick check

In scikit-learn's confusion matrix for classes [spam, ham], what does the top-right cell count?

Go deeper

1Build and interpret a confusion matrix for a model you've trained.
2Learn to identify which error type (false positive or false negative) matters more for your use case.
3Practice explaining a confusion matrix to a non-technical stakeholder.