Stop 02 · Math FoundationsTopics · 2/9

Calculus Essentials (Derivatives, Gradients)

In short

Models learn by asking one question again and again: if I nudge this number, does my error go up or down? Derivatives answer exactly that.

A derivative measures how a function changes; a gradient generalizes this to multiple variables — the core mechanism behind how models actually learn from data via gradient descent.

The mascot rides a scooter and glances down at its round speedometer.

Think of it like this

Your scooter's speedometer

The odometer shows total distance. The speedometer shows how fast that distance is changing right now. A derivative is a speedometer for any formula: how fast the output changes when you nudge the input a tiny bit.

Watch it happen

Slope of y = x² at five points

-2

-1

1/5These are x values. Our formula is y = x², a U-shaped curve.

The big ideas

What you need to know

Derivative = slope right here

It tells you how much the output changes per tiny step of the input, at one exact point.

Zero slope marks a flat spot

Minimums and maximums have slope zero. Training a model is hunting for that flat bottom.

Gradient = many slopes at once

With many inputs, the gradient is a vector of slopes. It points in the steepest uphill direction.

Chain rule links steps

If A affects B and B affects C, multiply their slopes. Neural networks use this to learn.

See it in code

Walk through the code

python

1def f(x):2    return x ** 23 4def slope(f, x, h=1e-5):5    return (f(x + h) - f(x - h)) / (2 * h)6 7for x in [-2, 0, 3]:8    print(x, round(slope(f, x), 3))   # -4.0, 0.0, 6.0

1/3Our formula: y = x squared.

You can estimate a derivative by nudging x a tiny bit both ways. The answers match the rule 2x.

Save yourself the pain

Common mistakes

Oops

Thinking you must solve long calculus problems by hand to do data science.

Do this instead

Understand what a derivative means. Libraries like PyTorch compute them for you automatically.

Oops

Reading the gradient as 'the way down'.

Do this instead

The gradient points uphill. To reduce error, models step in the opposite direction.

Your mission

Feel the slope

~15 min

Quick check

For a model's error, the derivative with respect to a weight is +8. What should you do to reduce the error?

Go deeper

1Review the basic definition of a derivative as a rate of change.
2Work through a simple gradient descent example on a single-variable function by hand.
3Learn just enough to intuitively understand 'the model moves downhill on the error surface'.