Stop 05 · Exploratory Data Analysis & Feature EngineeringTopics · 6/9

Feature Creation & Selection

In short

Good features often beat fancy algorithms. Creating smart columns from raw data, then dropping the useless ones, is where your domain sense makes models actually work.

Engineering new, more predictive features from raw data (e.g. 'days since last purchase' from a date column), and then selecting which features actually help versus add noise.

The mascot property dealer measures a flat's floor with a tape while holding a bunch of keys.

Think of it like this

The property dealer's eye

A dealer doesn't just note price and area. He works out price per square foot, the building's age and how far the metro is. And he ignores the colour of the curtains. That's feature creation and feature selection.

Watch it happen

From raw columns to a lean feature set

Raw

Create

Leak check

Select

Model

1/5Raw columns: price, area, bedrooms, year_built, listed_on date, metro distance.

The big ideas

What you need to know

Create from domain sense

Ratios, differences, date parts, counts per customer. Ask what a human expert would look at.

Watch for leakage

A feature that uses the answer, or future information, gives great scores that collapse in real use.

Select to simplify

Remove useless, duplicate or noisy columns. Simpler models are faster and easier to explain.

Selection methods

Filter by correlation, use model importances, or let L1 regularisation push weak features to zero.

See it in code

Walk through the code

python

1import pandas as pd2from sklearn.ensemble import RandomForestRegressor3 4df = pd.read_csv("flats.csv", parse_dates=["listed_on"])5df["price_per_sqft"] = df["price"] / df["area_sqft"]6df["building_age"] = 2026 - df["year_built"]7df["listed_month"] = df["listed_on"].dt.month8 9features = ["area_sqft", "bedrooms", "building_age", "listed_month", "metro_km"]10model = RandomForestRegressor(random_state=0).fit(df[features], df["price"])11imp = pd.Series(model.feature_importances_, index=features)12print(imp.sort_values(ascending=False))

1/4Useful for analysis, but it contains price, so it must not be a model input when predicting price.

Create features, keep the leaky one out of the model, then rank the rest by importance.

Save yourself the pain

Common mistakes

Oops

Including a feature that's only known after the outcome happens.

Do this instead

Ask for every feature: would I have this value at prediction time? If not, drop it.

Oops

Trusting feature importance as proof of cause.

Do this instead

Importance shows what the model used, not what causes the outcome. Correlated features also split importance.

Your mission

Engineer three features

~35 min

Quick check

Predicting loan default, a feature 'number_of_collection_calls' gives 99% accuracy. What's the likely problem?

Go deeper

1Create 3-5 new features from a raw dataset based on domain intuition.
2Learn a simple feature selection method (e.g. correlation with the target, feature importance).
3Practice explaining why a specific engineered feature should help the model.