Stop 02 · Math FoundationsTopics · 8/9

Linear Regression Math

In short

Linear regression is the 'hello world' of modelling, and its maths shows up everywhere later. Understand how it picks the best line and half of ML gets easier.

Understanding the math behind fitting a line — minimizing squared error between predictions and actual values — before treating it as a black-box function call in a library.

$The mascot stands next to a yellow and black auto-rickshaw, writing in a small notebook.$

Think of it like this

Working out auto fares

An auto charges a fixed base amount plus some rupees per kilometre. You don't know either number, but you have your last four rides and their fares. Linear regression finds the base and per-km rate that fit your rides best.

Watch it happen

Scoring one guess: fare = 30 + 13 × km

110

145

1/5Actual fares for rides of 2, 4, 6 and 9 km.

The big ideas

What you need to know

y = w·x + b

w is the slope (rupees per km), b is the intercept (base fare). With more features, w becomes a vector.

Least squares

The best line minimises the sum of squared errors between actual and predicted values.

Two ways to solve it

A direct formula, the normal equation, for small data, or gradient descent for very large data.

Assumptions behind it

A roughly straight-line relationship, independent errors and no wild outliers dominating the fit.

See it in code

Walk through the code

python

1import numpy as np2 3km = np.array([2, 4, 6, 9])4fare = np.array([60, 80, 110, 145])5 6X = np.column_stack([np.ones_like(km), km])   # [1, km] per ride7b, w = np.linalg.lstsq(X, fare, rcond=None)[0]8print(f"fare = {b:.1f} + {w:.1f} * km")       # about 33.7 + 12.4 * km

1/3Four rides: distance and what you paid.

NumPy solves least squares directly. The column of ones lets the model learn the base fare.

Save yourself the pain

Common mistakes

Oops

Reading a big slope as proof that x causes y.

Do this instead

Regression finds association. Causation needs an experiment or careful causal reasoning.

Oops

Forgetting the intercept.

Do this instead

Without b, the line must pass through zero. Most real relationships, like a base fare, don't.

Your mission

Fit your own fares

~20 min

Quick check

Why does least squares square the errors instead of just adding them?

Go deeper

1Derive (or follow a derivation of) the least-squares solution for simple linear regression.
2Fit a line by hand on a tiny dataset and compare it to a library's output.
3Learn to interpret a regression coefficient and intercept in plain language.