Stop 05 · Exploratory Data Analysis & Feature EngineeringTopics · 7/9

Dimensionality Reduction (PCA Overview)

In short

Fifty columns are hard to plot and can slow or confuse a model. PCA squeezes many related columns into a few new ones that keep most of the information.

A preview of Principal Component Analysis — reducing many correlated features into fewer, uncorrelated components that capture most of the original variance, covered in depth in Phase 8.

The mascot makes a bird-shaped hand shadow on a wall using a torch.

Think of it like this

Hand shadows on the wall

Your hand is 3D, its shadow is flat. Hold it at a bad angle and the shadow is just a blob. Turn it right and everyone sees a bird. PCA finds the angles that show the most shape, so a few dimensions keep most of the story.

Watch it happen

Variance kept by each component (a typical result)

0.36

0.19

0.11

0.07

…

1/5After scaling 13 columns, PCA creates 13 new components, sorted by how much variation each captures.

The big ideas

What you need to know

New axes along the spread

Each component is a direction where the data varies most, at right angles to the ones before.

Explained variance

explained_variance_ratio_ tells you how much variation each component keeps.

Scale first

PCA follows variance, so a column in lakhs would dominate one in single digits. Standardise before PCA.

Harder to explain

Components are mixes of original columns, so 'PC1 went up' is harder to explain to a manager.

See it in code

Walk through the code

python

1from sklearn.datasets import load_wine2from sklearn.decomposition import PCA3from sklearn.preprocessing import StandardScaler4 5X = StandardScaler().fit_transform(load_wine().data)   # 13 features6 7pca = PCA(n_components=0.90)         # keep 90% of the variance8X_small = pca.fit_transform(X)9print(X.shape, "->", X_small.shape)10print(pca.explained_variance_ratio_.round(2))

1/3Standardise all 13 wine measurements so each column counts equally.

Passing a fraction like 0.90 tells PCA to keep as many components as needed to reach 90% of the variance.

Save yourself the pain

Common mistakes

Oops

Running PCA on unscaled data.

Do this instead

Standardise first. Otherwise the column with the biggest numbers decides the components.

Oops

Using PCA when stakeholders need to know which original feature matters.

Do this instead

Prefer feature selection there. PCA components blend features and hide that link.

Your mission

Plot 13 dimensions in 2

~20 min

Quick check

PCA's first two components keep 55% of the variance. What does that mean?

Go deeper

1Read a conceptual (non-mathematical) explanation of what PCA does.
2Identify a dataset with many correlated features where PCA might help.
3Revisit Phase 8 for the full mechanics once you reach unsupervised learning.