Method
A/A test
Running a test with identical variants to validate the experiment setup and metrics.
12 cases from 11 companies apply A/A test. 3 were checked in an online A/B test; 5 report results in production.
Typical tasks: Measure impact without an A/B test, Get experiment results faster, Experiment when users affect each other, Decide who gets an offer and 4 more. Companies: LinkedIn, Stripe, Avito, Criteo, Thumbtack, Expedia Group and 5 more. Often combined with: Power analysis / MDE, Bootstrap, Nearest-neighbor matching, Geo experiment, Difference-in-differences and 25 more.
Validating Propensity Score Matching: Metrics and Tests![]() | ![]() | mobility | 2025-05 |
| How we tested the conversion impact of global payment methods | fintech | 2025-04 | |
| How We Launched a New Promotion Model: Matching and Regional Tests | classifieds | 2024-05 | |
| Privacy Sandbox Market Testing: a statistical model for measuring performance impacts | adtech | 2023-12 | |
| Accelerating Ranking Experimentation at Thumbtack with Interleaving | marketplace | 2023-07 | |
| A/B tests in KION | media-streaming | 2023-05 | |
| How to Size For Online Experiments With Ratio Metrics | travel | 2023-03 | |
| Ocelot: Scaling observational causal inference at LinkedIn | social | 2022-12 | |
| Faster ML Experimentation at Etsy with Interleaving | e-commerce | 2022-08 | |
| Measuring marketing incremental impacts beyond last click attribution | social | 2022-07 | |
| Geo-based AB Testing and Difference-in-Difference Analysis in Instacart Catalog | food-delivery | 2022-06 | |
| Artificial Counterfactual Estimation (ACE): Machine Learning-Based Causal Inference at Airbnb | travel | 2022-03 |
