Optimization Basics (Gradient Descent)

Summary

The iterative algorithm most ML models use to learn — repeatedly nudging parameters in the direction that reduces error, guided by the gradient from calculus.

• 1Trace a few iterations of gradient descent by hand on a simple function.
• 2Learn what the 'learning rate' controls, and what happens if it's set too high or too low.
• 3Understand the difference between batch, stochastic, and mini-batch gradient descent conceptually.