Stop 08 · Deep Learning ArchitecturesTopics · 11/11

Generative Adversarial Networks (GANs, Overview)

In short

A GAN pits two networks against each other: one makes fakes, one catches them. As each gets better, the fakes become startlingly real. GANs led the early wave of AI-generated faces.

Two networks (generator and discriminator) trained adversarially so the generator learns to produce realistic synthetic data — a historically important generative approach, now largely eclipsed by diffusion models for images.

The blob as a sweet shop owner taste-tests two plates of diamond-shaped sweets while a nervous cook watches.

Think of it like this

The new halwai vs the owner

A new cook at a mithai shop tries to copy the famous kaju katli. The owner taste-tests real and copied pieces and says which is which. The cook adjusts after every verdict. Eventually the owner can't tell. That's a GAN.

Watch it happen

One round of GAN training

Noise z

Generator

Discriminator

Real data

z ~ N(0, 1)

1/5Random noise goes into the generator. Different noise gives different outputs.

The big ideas

What you need to know

Generator: noise to data

It never sees real data directly. It learns only from the discriminator's feedback.

Discriminator: a binary judge

A classifier that outputs the probability that its input is real. Its gradients teach the generator.

A two-player game

They train in turns. If one gets far ahead of the other, learning stalls, which makes GANs tricky to train.

Diffusion took over images

Most text-to-image tools today use diffusion models, but GANs remain useful for fast generation and image-to-image tasks.

See it in code

Walk through the code

python

1import torch, torch.nn as nn2 3G = nn.Sequential(nn.Linear(16, 64), nn.ReLU(), nn.Linear(64, 2))   # noise -> fake point4D = nn.Sequential(nn.Linear(2, 64), nn.ReLU(), nn.Linear(64, 1))    # point -> real score5opt_g, opt_d = torch.optim.Adam(G.parameters(), 1e-3), torch.optim.Adam(D.parameters(), 1e-3)6bce = nn.BCEWithLogitsLoss()7 8real = torch.randn(64, 2) * 0.5 + 2           # the "real" data: points around (2, 2)9fake = G(torch.randn(64, 16))10 11# 1) Train D: real -> 1, fake -> 012d_loss = bce(D(real), torch.ones(64, 1)) + bce(D(fake.detach()), torch.zeros(64, 1))13opt_d.zero_grad(); d_loss.backward(); opt_d.step()14# 2) Train G: make D say 1 for fakes15g_loss = bce(D(fake), torch.ones(64, 1))16opt_g.zero_grad(); g_loss.backward(); opt_g.step()

1/4The generator maps 16 noise numbers to a 2D point. The discriminator scores a 2D point.

One training round on 2D points. Put the bottom half in a loop and the fake points drift towards (2, 2).

Save yourself the pain

Common mistakes

Oops

Forgetting fake.detach() when training the discriminator.

Do this instead

Without it, the discriminator's loss also pushes gradients into the generator, in the wrong direction.

Oops

Judging progress by the loss numbers alone.

Do this instead

GAN losses bounce around by design. Look at generated samples regularly to see real progress.

Your mission

A tiny GAN on points

~35 min

Quick check

In a GAN, what does the generator learn from?

Go deeper

1Learn the generator-vs-discriminator training loop conceptually.
2Understand why GANs are notoriously hard to train (mode collapse, instability).
3Read about how diffusion models have largely replaced GANs for image generation.