Stop 05 · Machine Learning FundamentalsTopics · 9/12

Naive Bayes

In short

Naive Bayes is tiny, fast and surprisingly good with text, like spam filters and simple review classifiers. It's also a neat way to see Bayes' rule doing real work.

A probabilistic classifier based on Bayes' theorem with a (naive) assumption of feature independence — fast, simple, and still competitive for text classification like spam detection.

The mascot squints suspiciously at a phone held at arm's length, a stack of envelopes beside it.

Think of it like this

Spotting a fake forward

A family group message says 'FREE recharge, click now, hurry'. Each word on its own makes you a bit more suspicious, and you add up those small suspicions to decide. Naive Bayes does that, treating each word as independent evidence.

Watch it happen

Scoring 'free recharge click now'

Prior

'free'

'click'

'meeting'

Verdict

1/5Start with the base rate: say 30% of messages in the training data were spam.

The big ideas

What you need to know

Bayes' rule underneath

It multiplies the prior by how likely each word is in each class, then compares classes.

The 'naive' assumption

It assumes words are independent given the class. Untrue, but it often works well anyway.

Very fast

Training is mostly counting, so it handles huge text datasets and makes a quick baseline.

Smoothing avoids zeros

A word never seen in spam would zero out everything. Laplace smoothing adds a small count to fix that.

See it in code

Walk through the code

python

1from sklearn.feature_extraction.text import CountVectorizer2from sklearn.naive_bayes import MultinomialNB3from sklearn.pipeline import make_pipeline4 5texts = ["win a free lottery now", "free recharge click here",6         "meeting moved to 3 pm", "notes for tomorrow's class",7         "click to claim free prize", "lunch at the canteen?"]8labels = [1, 1, 0, 0, 1, 0]   # 1 = spam9 10model = make_pipeline(CountVectorizer(), MultinomialNB())11model.fit(texts, labels)12print(model.predict(["free prize meeting", "class notes please"]))13print(model.predict_proba(["click for free recharge"]).round(2))

1/3Six tiny training messages and their labels.

CountVectorizer turns messages into word counts, and MultinomialNB learns which words lean spam. Real filters train on thousands of messages.

Save yourself the pain

Common mistakes

Oops

Trusting Naive Bayes probabilities as exact.

Do this instead

Its class choice is often good, but its probabilities tend to be overconfident. Calibrate them if you need real probabilities.

Oops

Using MultinomialNB on continuous measurements like height.

Do this instead

Use GaussianNB for continuous features. MultinomialNB is for counts, like words.

Your mission

Build a mini spam filter

~30 min

Quick check

What does the 'naive' in Naive Bayes refer to?

Go deeper

1Build a Naive Bayes spam classifier on a small text dataset.
2Learn why the independence assumption is 'naive' yet still works well in practice.
3Compare Multinomial vs. Gaussian Naive Bayes and when each applies.