Stop 10 · Computer VisionTopics · 8/9

Image Augmentation

In short

Your model has seen every training photo a dozen times and starts memorising them. Augmentation shows it a slightly different version each time, so it learns the object, not the exact photo.

Applying random transformations (flips, rotations, color jitter) to training images to artificially expand a dataset and improve generalization.

The blob bats in cricket nets as a friend throws balls from different angles in mixed light.

Think of it like this

Nets practice with varied throwdowns

A batter who only faces one bowler at one speed in perfect light gets out in the first real match. Good practice mixes angles, speeds and evening light. Augmentation does that for your model: flips, crops and lighting changes, every epoch.

Watch it happen

One photo, many training views

original

1/5Without augmentation, every epoch shows the model the exact same image.

The big ideas

What you need to know

Random, on the fly

Augmentations are applied fresh each time an image is loaded, so the dataset on disk never changes.

Label must still be true

Flipping a cat is fine. Flipping a '6' into something like a '9', or a left-turn sign, changes the meaning.

Training set only

Validation and test images get plain resize and normalise, so your scores stay honest and comparable.

Stronger tricks

MixUp and CutMix blend images and labels. RandAugment picks random policies. Useful when data is small.

See it in code

Walk through the code

python

1from torchvision import transforms2 3train_tf = transforms.Compose([4    transforms.RandomResizedCrop(224, scale=(0.7, 1.0)),5    transforms.RandomHorizontalFlip(),6    transforms.ColorJitter(brightness=0.3, contrast=0.3),7    transforms.RandomRotation(10),8    transforms.ToTensor(),9    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),10])11 12val_tf = transforms.Compose([13    transforms.Resize(256), transforms.CenterCrop(224), transforms.ToTensor(),14    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),15])

1/3Random crop between 70 and 100 percent of the area, and a 50 percent chance of a flip.

Random transforms for training, fixed ones for validation. Same normalisation in both.

Save yourself the pain

Common mistakes

Oops

Applying random augmentation to the validation set.

Do this instead

Validation should be deterministic. Otherwise your scores change run to run and mean little.

Oops

Using augmentations that break the label, like vertical flips for road scenes.

Do this instead

Ask 'would a real photo ever look like this?' If not, drop that transform.

Your mission

See what your model sees

~20 min

Quick check

You are training a model to read handwritten digits. Which augmentation is risky?

Go deeper

1Add an augmentation pipeline (e.g. `torchvision.transforms`) to a training run.
2Compare validation accuracy with and without augmentation on a small dataset.
3Learn which augmentations make sense for your specific task (don't flip text-containing images, for example).