Segmentation in Experiments

Summary

Checking whether an experiment's effect differs across user segments (new vs returning, mobile vs desktop) — a result that looks flat overall can hide a strong effect in one segment.

• 1Break down a hypothetical test result by 2-3 plausible segments.
• 2Learn why segment-level findings need larger samples to be trustworthy (multiple comparisons).
• 3Practice deciding which segment cuts are worth checking before you run the test, not after.