Stop 06 · Feature Engineering & Model EvaluationTopics · 10/12

Confusion Matrix

In short

One accuracy number hides what kind of mistakes your model makes. A confusion matrix lays every right and wrong answer out in a small table you can read in seconds.

A table of true/false positives and negatives — the raw material every classification metric is computed from, and the fastest way to see exactly where a model fails.

The mascot sorts ripe and raw mangoes into four baskets arranged in a square on a fruit stall.

Think of it like this

Sorting mangoes into four baskets

A fruit seller guesses ripe or raw for each mango, then cuts it open. Four baskets fill up: called ripe and was ripe, called ripe but was raw, called raw but was ripe, called raw and was raw. Two baskets are wins, two are different kinds of mistakes.

Watch it happen

Every prediction lands in one of four boxes

All predictions

Actually spam

Actually ham

TP: caught

FN: missed

FP: false alarm

TN: left alone

1/5Start with every email the model judged. First split by the real answer.

The big ideas

What you need to know

Rows are actual, columns predicted

That's the scikit-learn layout. Some tools flip it, so always check the axis labels before reading.

Diagonal means correct

Big numbers on the diagonal and small numbers elsewhere is what a good model looks like.

Every metric comes from it

Accuracy, precision, recall and F1 are all simple ratios of these four cells.

Works for many classes

With 5 classes you get a 5 by 5 grid. Off-diagonal cells show which classes get mixed up.

See it in code

Walk through the code

python

1from sklearn.metrics import confusion_matrix2 3y_true = ["spam", "ham", "spam", "ham", "ham", "spam", "ham", "ham"]4y_pred = ["spam", "ham", "ham", "ham", "spam", "spam", "ham", "ham"]5 6cm = confusion_matrix(y_true, y_pred, labels=["spam", "ham"])7print(cm)8# [[2 1]    actual spam: 2 caught, 1 missed9#  [1 4]]   actual ham: 1 false alarm, 4 correct

1/3Eight emails: the real label and the model's guess.

Passing labels fixes the row and column order, so you always know which cell is which.

Save yourself the pain

Common mistakes

Oops

Reading the matrix without checking which axis is actual and which is predicted.

Do this instead

Label your axes, or use ConfusionMatrixDisplay, which prints 'True label' and 'Predicted label'.

Oops

Judging a multi-class model only by its overall accuracy.

Do this instead

Look at off-diagonal cells to find which pairs of classes the model confuses, then fix those.

Your mission

Draw your own matrix

~20 min

Quick check

In scikit-learn's confusion matrix for classes [spam, ham], what does the top-right cell count?

Go deeper

1Generate and read a confusion matrix for a trained classifier.
2Learn to spot which type of error (false positive vs. false negative) matters more for your problem.
3Practice deriving precision and recall directly from the matrix.