Stop 08 · Unsupervised LearningTopics · 4/7

Principal Component Analysis (PCA)

In short

Fifty columns are impossible to plot and slow to model. PCA squeezes them into a few new columns that keep most of the information, so you can see and use your data.

A technique that transforms many correlated features into fewer uncorrelated components, ranked by how much variance they explain — used for dimensionality reduction and visualization.

The mascot holds a teapot in front of a lamp, casting a clear teapot shadow on the wall.

Think of it like this

The best shadow of a teapot

Hold a teapot in front of a lamp. Its 3D shape becomes a 2D shadow on the wall. From one angle the shadow is a useless blob. Turn it right and you can clearly see spout, handle and lid. PCA finds the angle that keeps the most detail.

Watch it happen

From 13 columns to 2

Standardise

Direction 1

Direction 2

Project

Check kept

1/5Centre each column at zero and scale it, so no column wins just because of its units.

The big ideas

What you need to know

New columns, not chosen ones

Each component is a weighted mix of all original features. PCA doesn't just pick a few columns.

Variance kept

Keep enough components to retain, say, 90 to 95% of the variance, or use 2 or 3 just for plotting.

Always scale first

PCA chases spread. A column in lakhs would dominate one in single digits unless you standardise.

Harder to explain

'Component 1' is a blend of features. Look at its loadings to describe what it roughly means.

See it in code

Walk through the code

python

1from sklearn.datasets import load_wine2from sklearn.decomposition import PCA3from sklearn.pipeline import make_pipeline4from sklearn.preprocessing import StandardScaler5 6X, y = load_wine(return_X_y=True)        # 13 features7 8pca = PCA(n_components=2)9X2 = make_pipeline(StandardScaler(), pca).fit_transform(X)10 11print(X2.shape)                           # (178, 2)12print(pca.explained_variance_ratio_.round(2))

1/3178 wines, 13 measured features each.

13 chemical measurements of wines become 2 components you can scatter-plot, coloured by wine type.

Save yourself the pain

Common mistakes

Oops

Running PCA on unscaled columns with different units.

Do this instead

Standardise first, or the largest-unit column becomes component 1 by itself.

Oops

Fitting PCA on the full dataset before the train/test split.

Do this instead

Fit PCA on training data only, inside a Pipeline, then transform the test set.

Your mission

See the wine types

~20 min

Quick check

What is the first principal component?

Go deeper

1Apply PCA to a high-dimensional dataset and plot the first 2 components.
2Learn to interpret an explained-variance plot to decide how many components to keep.
3Understand PCA components lose direct interpretability — a real trade-off to weigh.