Stop 10 · Computer VisionTopics · 7/9

Transfer Learning for Vision

In short

You don't need a million photos to build a good image classifier. Borrow a model that already learned to see, and teach it only the last step. A few hundred images can be enough.

Fine-tuning a pretrained vision backbone on your own images — the standard, data-efficient way to build a working image classifier without training from scratch.

The blob cook chops vegetables with practised skill while a dosa cooks on a new pan.

Think of it like this

The cook who switches restaurants

A cook moves from a Punjabi dhaba to a South Indian restaurant. He doesn't relearn chopping, tempering or handling heat. He only learns the new dishes. A pretrained model already knows edges, textures and shapes; you just teach it your classes.

Watch it happen

Fine-tuning a pretrained backbone

Pretrained

Freeze

New head

Train head

Unfreeze

1/5Load ResNet-50 with ImageNet weights. Early layers detect edges, later ones detect parts and objects.

The big ideas

What you need to know

Features are reusable

Edges, corners and textures are useful for almost any photo task, so early layers transfer well.

Feature extraction vs fine-tuning

Freezing everything except the head is quick and safe. Unfreezing more layers adapts better but can overfit small data.

Small learning rates

When you unfreeze the backbone, use a learning rate around 10 times smaller so you don't wreck the pretrained weights.

Match the preprocessing

Use the same resize and normalisation the pretrained model was trained with.

See it in code

Walk through the code

python

1import torch.nn as nn2from torch.optim import AdamW3from torchvision.models import resnet50, ResNet50_Weights4 5model = resnet50(weights=ResNet50_Weights.DEFAULT)6 7for p in model.parameters():8    p.requires_grad = False                       # freeze the backbone9 10model.fc = nn.Linear(model.fc.in_features, 4)     # new head: 4 classes11 12optimizer = AdamW(model.fc.parameters(), lr=1e-3)13loss_fn = nn.CrossEntropyLoss()14# ...normal training loop over your DataLoader...

1/4Load ImageNet-pretrained weights. This is all the knowledge you are borrowing.

Freeze ResNet-50, swap the final layer for your 4 classes and train only that layer first.

Save yourself the pain

Common mistakes

Oops

Unfreezing the whole network on day one with lr=0.01.

Do this instead

Train the head first, then unfreeze top layers with a small learning rate like 1e-4 or lower.

Oops

Skipping a validation set because training accuracy looks great.

Do this instead

Hold out 15 to 20 percent of images. Small datasets overfit fast and only validation shows it.

Your mission

Four-class classifier in an hour

~45 min

Quick check

You have 300 labelled images. What is the best first approach?

Go deeper

1Fine-tune a pretrained CNN or ViT on a small custom image dataset.
2Practice freezing early layers and only training the final classification head first.
3Compare results from freezing vs. fully fine-tuning the backbone.