Batch Normalization

Summary

A technique that normalizes layer inputs during training, stabilizing and speeding up convergence in deep networks.

• 1Train a deep network with and without batch norm and compare training stability.
• 2Learn where batch norm layers are typically placed relative to activations.
• 3Understand the difference in batch norm behavior between training and inference mode.