Stop 06 · Machine Learning FundamentalsTopics · 7/13

Logistic Regression

In short

Despite the name, logistic regression is for yes/no questions: will this user churn, is this payment fraud. It gives you a probability, not just a label, which is often what the business really needs.

A close cousin of linear regression used for classification — outputs a probability rather than a raw number, and remains a strong, interpretable baseline for binary classification problems.

The mascot at an office desk sorts student files into two trays.

Think of it like this

The placement cell's guess

Looking at CGPA, internships and projects, the placement officer says 'about 80% chance this student gets placed'. Then the college decides: above 50%, normal prep; below, extra coaching. Logistic regression gives that probability, and you pick the cut-off.

Watch it happen

From score to probability to decision

-3

-1

1/4The model first computes a score: a weighted sum of features, like linear regression. It can be any number.

The big ideas

What you need to know

Outputs probabilities

predict_proba gives the chance of each class. predict just applies a 0.5 cut-off.

Choose the threshold

Pick the cut-off based on what false alarms and misses cost the business.

Readable weights

Positive weights push towards 'yes'. Each weight changes the log-odds by a fixed amount.

Scale your features

Its default regularisation works best when features are on similar scales.

See it in code

Walk through the code

python

1from sklearn.datasets import load_breast_cancer2from sklearn.model_selection import train_test_split3from sklearn.pipeline import make_pipeline4from sklearn.preprocessing import StandardScaler5from sklearn.linear_model import LogisticRegression6 7X, y = load_breast_cancer(return_X_y=True)8X_tr, X_te, y_tr, y_te = train_test_split(X, y, stratify=y, random_state=0)9 10model = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000))11model.fit(X_tr, y_tr)12proba = model.predict_proba(X_te)[:, 1]      # probability of class 113print(proba[:5].round(3))14print((proba >= 0.3).astype(int)[:5])        # a lower, more cautious cut-off

1/4Load data and make a stratified train-test split.

Get probabilities first, then apply the threshold that suits your problem, rather than accepting 0.5 by default.

Save yourself the pain

Common mistakes

Oops

Using logistic regression's predict and never looking at probabilities.

Do this instead

Use predict_proba and choose a threshold that matches the business cost of each mistake.

Oops

Thinking 'regression' means it predicts numbers like prices.

Do this instead

It predicts class probabilities. For prices, use linear regression or other regressors.

Your mission

Move the threshold

~25 min

Quick check

Missing a real fraud costs far more than checking a genuine payment. How should you change the threshold?

Go deeper

1Fit a logistic regression model on a binary classification dataset.
2Learn to interpret coefficients in terms of odds ratios.
3Compare its performance to a simple baseline (always predicting the majority class).