Stop 06 · Feature Engineering & Model EvaluationTopics · 2/12

One-Hot & Label Encoding

In short

Models only understand numbers, but your data says 'Delhi', 'Pune' and 'Kochi'. How you turn words into numbers can quietly teach the model something false.

Turning categorical variables into numeric form models can consume — one-hot for unordered categories, label/ordinal encoding for ordered ones.

The blob ticks one box in a row of empty checkboxes on a big paper form.

Think of it like this

Checkboxes on a form

If a form asks you to write 1 for Delhi, 2 for Pune and 3 for Kochi, a model may think Kochi is 'three times' Delhi. A form with a separate tick box for each city avoids that. That tick box idea is one-hot encoding.

Watch it happen

Encoding the 'city' column

Label / ordinal

One column: 0, 1, 2
Implies an order and gaps
Good for S < M < L

One-hot

One 0/1 column per value
No fake order between cities
Many columns if many values

Categories with a real order (size, education level) get ordinal codes. Categories without one (city, colour) get one-hot. Trees tolerate either better than linear models.

The big ideas

What you need to know

One-hot: a column per value

City becomes city_Delhi, city_Kochi, city_Pune. Each row has a 1 in exactly one of them and 0 in the rest.

Ordinal for real order

Small, medium, large have a natural order, so 0, 1, 2 makes sense. Tell the encoder the order explicitly.

LabelEncoder is for the target

In scikit-learn, LabelEncoder is meant for y, the answer column. Use OrdinalEncoder or OneHotEncoder for input features.

Plan for unseen values

A new city may show up in production. handle_unknown='ignore' gives it all zeros instead of crashing.

See it in code

Walk through the code

python

1import pandas as pd2from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder3 4df = pd.DataFrame({"city": ["Delhi", "Pune", "Delhi", "Kochi"],5                   "size": ["S", "L", "M", "S"]})6 7ohe = OneHotEncoder(sparse_output=False, handle_unknown="ignore")8print(ohe.fit_transform(df[["city"]]))   # one 0/1 column per city9 10ord_enc = OrdinalEncoder(categories=[["S", "M", "L"]])11print(ord_enc.fit_transform(df[["size"]]).ravel())  # [0. 2. 1. 0.]

1/3A tiny table with one unordered column (city) and one ordered column (size).

City has no order, so it gets three 0/1 columns. Size has a real order, so it gets 0, 1, 2 in the order you specify.

Save yourself the pain

Common mistakes

Oops

Label encoding a city column and feeding it to linear regression.

Do this instead

Linear models read 0, 1, 2 as amounts. One-hot encode unordered categories for them.

Oops

One-hot encoding a column with 10,000 unique values like user_id.

Do this instead

IDs usually carry no general pattern. Drop them, or group rare values into 'other' before encoding.

Your mission

Encode a real table

~20 min

Quick check

You encode colour as red=0, green=1, blue=2 and train linear regression. What's the risk?

Go deeper

1Practice both encoding types with pandas `get_dummies` and scikit-learn encoders.
2Learn when one-hot encoding blows up dimensionality and what to do about it.
3Understand target/mean encoding as an alternative for high-cardinality categoricals.