Method

t-test

Mean comparison test, including Welch and cluster-robust variants.

12 cases from 11 companies apply t-test. 4 were checked in an online A/B test; 5 report results in production.

Typical tasks: Get experiment results faster, Experiment when users affect each other, Avoid false wins in A/B tests, Evaluate models and 3 more. Companies: Expedia Group, Pinterest, DoorDash, Avito, Grab, The New York Times and 5 more. Often combined with: A/B test, Bootstrap, Power analysis / MDE, Interleaving, Geo experiment and 27 more.