Experiment Design & Sample Size

Summary

Deciding what to measure, how long to run a test, and how many users you need before you can trust the result — under-designed experiments produce confident-sounding, wrong conclusions.

• 1Practice calculating (or estimating) required sample size for a small hypothetical test.
• 2Learn why low-traffic features are hard to A/B test reliably.
• 3Understand the trade-off between test duration and time-to-decision.