Stop 09 · Natural Language ProcessingTopics · 5/10

Sentiment Analysis

In short

Businesses get thousands of reviews and comments every day, far too many to read. Sentiment analysis sorts them into happy, unhappy and neutral, so the angry ones get noticed fast.

Classifying text by expressed sentiment (positive/negative/neutral) — one of the most common practical NLP tasks and a good first classification project.

The blob reads a thick feedback register in a mess hall and sorts notes into happy and sad piles.

Think of it like this

The hostel mess feedback register

The mess manager can't read every complaint in the feedback register, so he skims each line and sorts it: happy, angry, or 'meh'. If 'angry' suddenly spikes on Tuesdays, he checks Tuesday's menu. A sentiment model does that sorting at scale.

Watch it happen

What happens to one review

Review

Tokenise

Model

Scores

Label

1/5Input: 'Not bad at all for the price!' The word 'bad' appears, but the meaning is positive.

The big ideas

What you need to know

Word lists: quick and blunt

Lexicon tools like VADER count positive and negative words. Fast and explainable, but they miss sarcasm and context.

Transformers read context

Fine-tuned models like DistilBERT handle negation ('not bad') far better because they read words together.

Aspect-based sentiment

'Food great, delivery terrible' is mixed. Aspect models score each topic separately, which is often what businesses need.

Domain shift

A model trained on movie reviews may misread cricket comments or Hinglish. Test on your real data.

See it in code

Walk through the code

python

1from transformers import pipeline2 3clf = pipeline("sentiment-analysis",4               model="distilbert/distilbert-base-uncased-finetuned-sst-2-english")5 6reviews = [7    "The paneer was fresh and the service was quick.",8    "Waited 50 minutes and the food came cold.",9    "Not bad at all for the price!",10]11for review, result in zip(reviews, clf(reviews)):12    print(result["label"], round(result["score"], 2), "|", review)

1/3Load a DistilBERT model fine-tuned on movie review sentences. It downloads once, then loads from cache.

A ready-made English sentiment model in three lines. It only knows POSITIVE and NEGATIVE, with no neutral class.

Save yourself the pain

Common mistakes

Oops

Forcing every review into positive or negative.

Do this instead

Plenty of text is neutral or mixed. Use a model with a neutral class, or treat low-confidence results as neutral.

Oops

Running an English-only model on Hindi or Hinglish comments.

Do this instead

Use a multilingual model, or fine-tune on labelled examples in the language mix your users actually write.

Your mission

Test it on tricky text

~20 min

Quick check

Why does a transformer model usually beat a simple word-count approach on 'The movie was not bad'?

Go deeper

1Build a sentiment classifier with TF-IDF + logistic regression as a baseline.
2Then fine-tune a pretrained transformer model and compare accuracy.
3Test your model on ambiguous or sarcastic text to see where it breaks.