Activation Functions

Summary

Non-linear functions (ReLU, sigmoid, tanh, softmax) applied after each layer — without them, stacking layers would collapse into a single linear transformation.

• 1Plot ReLU, sigmoid, tanh and softmax to see their shapes and ranges.
• 2Learn why ReLU is the default choice for hidden layers (avoids vanishing gradients).
• 3Understand which activation belongs at the output layer for regression vs. classification.