Statistical Significance Basics

Summary

Understanding whether a difference between two numbers (e.g. two A/B test variants) is a real effect or just random noise — enough intuition to avoid shipping a 'winner' that isn't actually one.

• 1Learn what a p-value intuitively represents, without the full formula derivation.
• 2Understand why small sample sizes produce unreliable 'wins'.
• 3Learn the term 'peeking' and why stopping a test early on a good result is a trap.