Stop 05 · Exploratory Data Analysis & Feature EngineeringTopics · 4/9

Feature Scaling (Normalization / Standardization)

In short

Area in square feet runs into thousands, bedrooms are single digits. Many models get confused by that difference and treat bigger numbers as more important. Scaling fixes it.

Rescaling numeric features to comparable ranges — normalization maps to a 0-1 range, standardization centers around a mean of 0 with unit variance — required for many algorithms (like KNN, SVM, gradient descent-based models) to work correctly.

The mascot in a kitchen holds a measuring cup in one hand next to a weighing scale.

Think of it like this

Two aunties, two units

One aunty writes her rice recipe in cups, the other in grams. Comparing '2' with '300' tells you nothing until you convert both to one unit. Feature scaling puts every column on a comparable scale before the model compares them.

Watch it happen

One column, two kinds of scaling

650

1200

2400

1/4Flat areas in square feet. Next to bedrooms (2, 3, 4), these numbers are huge.

The big ideas

What you need to know

Min-max to 0 to 1

Good when you need a fixed range. Sensitive to outliers, since one extreme value squashes the rest.

Standardise to z-scores

Subtract the mean, divide by standard deviation. A common default for linear models, SVMs and neural networks.

Who needs it

Distance and gradient based models: KNN, SVM, k-means, PCA, regularised regression, neural nets. Trees mostly don't.

Fit on train only

Learn the mean and std from training data, then apply those same numbers to test data.

See it in code

Walk through the code

python

1import numpy as np2from sklearn.preprocessing import MinMaxScaler, StandardScaler3from sklearn.model_selection import train_test_split4 5X = np.array([[650, 2], [1200, 3], [2400, 4], [900, 2], [1500, 3]])  # sqft, bedrooms6X_train, X_test = train_test_split(X, test_size=0.4, random_state=0)7 8scaler = StandardScaler().fit(X_train)     # learn mean and std from train only9X_train_s = scaler.transform(X_train)10X_test_s = scaler.transform(X_test)        # reuse the same numbers on test11 12print(MinMaxScaler().fit_transform(X))     # every column squeezed into 0 to 1

1/3Two features on very different scales, split into train and test.

fit learns the scaling numbers, transform applies them. Fitting only on training data avoids leaking test information.

Save yourself the pain

Common mistakes

Oops

Scaling the whole dataset before splitting into train and test.

Do this instead

Split first, fit the scaler on train only. Otherwise test data leaks into training.

Oops

Scaling features for a random forest and expecting better results.

Do this instead

Tree models split on thresholds, so scaling rarely changes them. Spend the effort elsewhere.

Your mission

See scaling change KNN

~25 min

Quick check

Why must you fit the scaler on training data only?

Go deeper

1Apply both normalization and standardization to the same numeric column and compare.
2Learn which model types actually require scaled features (distance-based, gradient-based) and which don't (tree-based).
3Practice fitting a scaler on training data only, then applying it to test data — never the reverse.