Stop 07 · Deep Learning FoundationsTopics · 7/11

PyTorch Basics

In short

PyTorch is the library most AI research and many production models are written in. Learn tensors, autograd and the five-line training loop, and you can read most model code out there.

The dominant deep learning framework in research and increasingly in production — tensors, autograd, `nn.Module` and the training loop pattern you'll use for almost every DL project.

The blob behind a kirana shop counter writes in a thick ledger book next to jars of sweets.

Think of it like this

The kirana store's khata

The kirana uncle writes every sale in his khata book. At month end, he can trace exactly which items led to the total you owe. PyTorch keeps a similar record of every operation on a tensor, so it can trace back and work out gradients.

Watch it happen

The training loop

zero_grad

Forward

Loss

backward

step

1/5Clear the gradients from the last step, because PyTorch adds new ones on top of old ones.

The big ideas

What you need to know

Tensors

A tensor is a NumPy-like array that can live on a GPU and track gradients. Shapes like (batch, features) matter everywhere.

Autograd

Set requires_grad=True, do maths, call .backward(). PyTorch fills in .grad for you.

nn.Module

Models are classes built from layers like nn.Linear. model.parameters() hands every weight to the optimiser.

DataLoader

It batches and shuffles your data so the loop gets a fresh mini-batch on every step.

See it in code

Walk through the code

python

1import torch2import torch.nn as nn3 4X = torch.linspace(0, 1, 100).unsqueeze(1)    # 100 rows, 1 feature5y = 3 * X + 1 + 0.05 * torch.randn_like(X)    # true line: y = 3x + 16 7model = nn.Linear(1, 1)8opt = torch.optim.SGD(model.parameters(), lr=0.5)9 10for epoch in range(200):11    opt.zero_grad()                                 # clear old gradients12    loss = nn.functional.mse_loss(model(X), y)      # forward pass + loss13    loss.backward()                                 # autograd fills .grad14    opt.step()                                      # update weight and bias15 16print(model.weight.item(), model.bias.item())  # close to 3 and 1

1/3Make 100 points on the line y = 3x + 1 with a little noise. unsqueeze makes the shape (100, 1).

A model with one weight and one bias learns a straight line. Bigger models use the exact same loop.

Save yourself the pain

Common mistakes

Oops

Mixing up shapes, like passing a (100,) tensor where (100, 1) is expected.

Do this instead

Print .shape often. Most PyTorch errors in your first month are shape mismatches.

Oops

Running evaluation with gradients on.

Do this instead

Wrap evaluation in torch.no_grad() and call model.eval(). It saves memory and switches off training-only layers.

Your mission

Your first training loop

~30 min

Quick check

What does loss.backward() do in PyTorch?

Go deeper

1Work through PyTorch's official 60-minute blitz tutorial.
2Build and train a small feedforward network from scratch with `nn.Module`.
3Learn to move tensors and models between CPU and GPU correctly.