Stop 06 · Machine Learning FundamentalsTopics · 13/13

Naive Bayes

In short

Naive Bayes is tiny, fast and surprisingly good with text, like spam filters and simple review classifiers. It's also a neat way to see Bayes' rule doing real work.

A fast, simple classifier based on Bayes' theorem, assuming features are independent (an assumption that's often technically wrong but works surprisingly well in practice) — especially strong for text classification.

The mascot squints suspiciously at a phone held at arm's length, a stack of envelopes beside it.

Think of it like this

Spotting a fake forward

A family group message says 'FREE recharge, click now, hurry'. Each word on its own makes you a bit more suspicious, and you add up those small suspicions to decide. Naive Bayes does that, treating each word as independent evidence.

Watch it happen

Scoring 'free recharge click now'

Prior

'free'

'click'

'meeting'

Verdict

1/5Start with the base rate: say 30% of messages in the training data were spam.

The big ideas

What you need to know

Bayes' rule underneath

It multiplies the prior by how likely each word is in each class, then compares classes.

The 'naive' assumption

It assumes words are independent given the class. Untrue, but it often works well anyway.

Very fast

Training is mostly counting, so it handles huge text datasets and makes a quick baseline.

Smoothing avoids zeros

A word never seen in spam would zero out everything. Laplace smoothing adds a small count to fix that.

See it in code

Walk through the code

python

1from sklearn.feature_extraction.text import CountVectorizer2from sklearn.naive_bayes import MultinomialNB3from sklearn.pipeline import make_pipeline4 5texts = ["win a free lottery now", "free recharge click here",6         "meeting moved to 3 pm", "notes for tomorrow's class",7         "click to claim free prize", "lunch at the canteen?"]8labels = [1, 1, 0, 0, 1, 0]   # 1 = spam9 10model = make_pipeline(CountVectorizer(), MultinomialNB())11model.fit(texts, labels)12print(model.predict(["free prize meeting", "class notes please"]))13print(model.predict_proba(["click for free recharge"]).round(2))

1/3Six tiny training messages and their labels.

CountVectorizer turns messages into word counts, and MultinomialNB learns which words lean spam. Real filters train on thousands of messages.

Save yourself the pain

Common mistakes

Oops

Trusting Naive Bayes probabilities as exact.

Do this instead

Its class choice is often good, but its probabilities tend to be overconfident. Calibrate them if you need real probabilities.

Oops

Using MultinomialNB on continuous measurements like height.

Do this instead

Use GaussianNB for continuous features. MultinomialNB is for counts, like words.

Your mission

Build a mini spam filter

~30 min

Quick check

What does the 'naive' in Naive Bayes refer to?

Go deeper

1Fit a Naive Bayes classifier on a simple text classification dataset.
2Learn why the 'naive' independence assumption rarely holds, yet the model still performs well.
3Compare its speed to a more complex model on the same dataset.