Method

A/A test

Running a test with identical variants to validate the experiment setup and metrics.

12 cases from 11 companies apply A/A test. 3 were checked in an online A/B test; 5 report results in production.

Typical tasks: Measure impact without an A/B test, Get experiment results faster, Experiment when users affect each other, Decide who gets an offer and 4 more. Companies: LinkedIn, Stripe, Avito, Criteo, Thumbtack, Expedia Group and 5 more. Often combined with: Power analysis / MDE, Bootstrap, Nearest-neighbor matching, Geo experiment, Difference-in-differences and 25 more.