Sample Size & Test Duration

Summary

Calculating how many users (and how long) you need before a test has enough statistical power to detect a real effect — running a test too briefly is one of the most common experimentation mistakes.

• 1Use a free sample size calculator for a hypothetical test scenario.
• 2Learn how baseline conversion rate and minimum detectable effect both affect required sample size.
• 3Understand why low-traffic features are especially hard to test reliably.