Stop 04 · Statistics for AnalystsTopics · 9/11

Regression Basics

In short

"How much does each extra rupee of ad spend bring in?" Regression puts a number on relationships like that, and it's the doorway to most predictive work.

A statistical method for modeling the relationship between a dependent variable and one or more predictors — the simplest form of 'predicting' a number from other numbers.

The jaadu holds a long ruler against a wall, drawing a line through scattered paint dots.

Think of it like this

The best straight line through paint dots

Splash paint dots on a wall, then lay a long ruler so it passes as close as possible to all of them. That ruler is a regression line. Its slope says how much one thing rises when the other goes up by one.

Watch it happen

Fitting a line to ad spend and sales

10k: 52

20k: 61

30k: 75

40k: 79

50k: 93

1/5Five weeks of ad spend and the orders that followed, in hundreds. The dots trend upward.

The big ideas

What you need to know

Slope and intercept

y = intercept + slope × x. The slope is the change in y for one unit of x.

R-squared

Share of variation in y explained by the model, from 0 to 1. High doesn't mean causal.

Multiple regression

Adds more predictors, like price and season, to separate their effects.

Check the leftovers

Plot residuals, the gaps from the line. Patterns there mean a straight line doesn't fit.

See it in code

Walk through the code

python

1import pandas as pd2import statsmodels.formula.api as smf3 4df = pd.DataFrame({5    "ad_spend": [10, 20, 30, 40, 50],6    "orders":   [52, 61, 75, 79, 93],7})8 9model = smf.ols("orders ~ ad_spend", data=df).fit()10print(model.params)11print(round(model.rsquared, 3))

1/3Ad spend in thousands of rupees and orders in hundreds, for five weeks.

A simple linear regression with statsmodels. The slope on ad_spend comes out at about 1.0.

Save yourself the pain

Common mistakes

Oops

Reading a regression slope as proof that X causes Y.

Do this instead

Regression on observational data shows association. Confounders can still drive both.

Oops

Predicting far outside the range you trained on.

Do this instead

Only trust predictions near your data. Beyond that, the relationship may bend or break.

Your mission

Draw your own trendline

~20 min

Quick check

A regression gives marks = 35 + 4 × hours_studied. What does the 4 mean?

Go deeper

1Plot a simple scatter plot and fit a line by eye before learning the formal method.
2Learn to interpret a regression coefficient in plain language.
3Understand R-squared as 'how much variance this model explains', roughly.