Stop 02 · Math & Statistics for AITopics · 9/13

Bayes' Theorem

In short

A test is 99% accurate and says you're positive. Are you 99% likely to have it? Often not even close. Bayes' theorem shows why, and it's the logic behind spam filters and Naive Bayes.

The formula for updating a belief given new evidence — the foundation of Naive Bayes classifiers and a core mental model for reasoning about model uncertainty.

The purple blob stands on a wet street with a closed umbrella, a water tanker parked nearby.

Think of it like this

The wet road in the morning

You step out and the road is wet. Did it rain? Maybe, but the municipal tanker also washes this road most mornings. Before deciding, you weigh how often each one happens. Bayes does exactly that: start with how common something is, then update with the clue.

Watch it happen

1,000 emails and the word 'free'

1,000 emails

200 spam

800 not spam

160 say 'free'

40 don't

40 say 'free'

760 don't

1/4Prior: 20% of all emails are spam. That's P(spam) = 0.2 before reading anything.

The big ideas

What you need to know

The formula

P(A given B) = P(B given A) × P(A) / P(B). Posterior = likelihood × prior / evidence.

Think in counts

Picture 1,000 people or emails and split them. Counting is easier and less error-prone than juggling fractions.

Rare things stay rare

If only 1% have a disease, even a good test gives many false alarms. The prior matters a lot.

Naive Bayes uses it per word

Spam filters multiply the evidence from each word, assuming words are independent. Naive, but it works surprisingly well.

See it in code

Walk through the code

python

1def bayes(prior, p_clue_if_true, p_clue_if_false):2    evidence = p_clue_if_true * prior + p_clue_if_false * (1 - prior)3    return p_clue_if_true * prior / evidence4 5# spam filter: 20% spam, 'free' in 80% of spam, 5% of normal mail6print(bayes(0.20, 0.80, 0.05))      # 0.87 8# medical test: 1% have it, test catches 99%, 5% false alarms9print(round(bayes(0.01, 0.99, 0.05), 3))   # 0.167

1/3Evidence is how often the clue shows up at all, from true cases and false alarms together.

Same formula, very different answers. With a rare condition, a positive test only means about a 1 in 6 chance.

Save yourself the pain

Common mistakes

Oops

Confusing P(positive given sick) with P(sick given positive).

Do this instead

They're different questions. Bayes converts one into the other using the prior.

Oops

Ignoring how rare the thing is in the first place.

Do this instead

Always start with the base rate. A tiny prior can make a strong clue much weaker than it looks.

Your mission

Build a one-word spam filter

~20 min

Quick check

Only 2 in 1,000 transactions are fraud. A detector flags 90% of fraud and 1% of normal ones. Roughly what share of flagged transactions are actually fraud?

Go deeper

1Work through a classic Bayes' theorem example (e.g. medical test false positives).
2Implement a simple Naive Bayes spam classifier by hand.
3Learn the terms: prior, likelihood, posterior, evidence.