Method

Multiple-testing correction

FDR, Bonferroni and similar corrections for many metrics or variants.

16 cases from 13 companies apply Multiple-testing correction. 1 was checked in an online A/B test; 3 report results in production. Includes variants: FDR correction.

Typical tasks: Avoid false wins in A/B tests, Get experiment results faster, Detect anomalies, Experiment when users affect each other and 4 more. Companies: Spotify, DoorDash, Pinterest, Canva, Grab, Square and 7 more. Often combined with: FDR correction, A/B test, Power analysis / MDE, Bayesian A/B test, t-test and 32 more.