Stop 06 · Feature Engineering & Model EvaluationTopics · 1/12

Feature Scaling & Normalization

In short

Income in lakhs and age in years sit side by side in your table. To some models, the bigger number simply shouts louder. Scaling makes every column speak at the same volume.

Rescaling features to comparable ranges (standardization, min-max scaling) — required for distance-based and gradient-based algorithms to converge and perform correctly.

The blob balances two different-sized stacks of answer sheets on a weighing scale until they level out.

Think of it like this

Marks from different boards

One friend scored 450 out of 500, another 520 out of 600. You can't compare raw totals, so you convert both to percentages. Scaling does the same for features: it puts every column on a common scale before the model compares them.

Watch it happen

Min-max scaling monthly income

120

1/4Five students' family income in thousands of rupees. The range runs from 20 to 120.

The big ideas

What you need to know

Min-max squeezes to 0..1

MinMaxScaler maps the smallest value to 0 and the largest to 1. One extreme outlier can squash everything else together.

Standardising centres on 0

StandardScaler subtracts the mean and divides by the standard deviation, so each column has mean 0 and spread 1.

Who needs it

KNN, SVMs, k-means, PCA and anything trained with gradient descent care about scale. Decision trees and random forests don't.

Learn the scale from train only

Fit the scaler on training rows, then apply it to test rows. Otherwise test information leaks into training.

See it in code

Walk through the code

python

1import numpy as np2from sklearn.preprocessing import MinMaxScaler, StandardScaler3 4# columns: monthly income (Rs), age (years)5X = np.array([[20000, 21], [35000, 25], [50000, 30], [120000, 45]])6 7print(MinMaxScaler().fit_transform(X).round(2))8# every column now runs from 0 to 19 10print(StandardScaler().fit_transform(X).round(2))11# every column now has mean 0 and std 1

1/3Two columns with wildly different units: rupees in the tens of thousands, ages in the twenties.

Before scaling, income differences are thousands of times bigger than age differences. After scaling, both columns carry similar weight.

Save yourself the pain

Common mistakes

Oops

Scaling the whole dataset first, then splitting into train and test.

Do this instead

Split first, fit the scaler on training data, then transform both. A Pipeline handles this for you.

Oops

Scaling the target column out of habit and then reading predictions as rupees.

Do this instead

Scale features, not the target, unless you have a reason. If you do scale it, invert the scaling before reporting.

Your mission

Watch KNN care about scale

~20 min

Quick check

Which model's results would change the least if you skipped feature scaling?

Go deeper

1Practice StandardScaler and MinMaxScaler in scikit-learn.
2Learn which algorithms actually need scaling (KNN, SVM, neural nets) vs. which don't (tree-based models).
3Fit scalers only on training data and apply them to test data to avoid leakage.