Method
Regression adjustment
Covariate-adjusted treatment effect (ANCOVA, Lin's estimator).
6 cases from 4 companies apply Regression adjustment. 1 was checked in an online A/B test; 4 report results in production.
Typical tasks: Get experiment results faster, Avoid false wins in A/B tests, Measure impact without an A/B test, Detect anomalies and 1 more. Companies: DoorDash, Instacart, Lyft and 1 more. Often combined with: Power analysis / MDE, Covariate balance check, Linear regression, Multiple-testing correction, Backdoor adjustment and 12 more.
| Unlocking 66% Faster Experiments with Covariate Adjustment: Power Analysis for Clustered Data, Ratio Metrics, and OLS Regression | marketplace | 2024-06 | |
| Addressing the Challenges of Sample Ratio Mismatch in A/B Testing | food-delivery | 2023-10 | |
| Deep Dive Into Variance Reduction | general | 2022-11 | |
| Using Back-Door Adjustment Causal Analysis to Measure Pre-Post Effects | food-delivery | 2022-06 | |
| Meet Dash-AB: The Statistics Engine of Experimentation at DoorDash | food-delivery | 2022-05 | |
| Challenges in Experimentation | mobility | 2022-04 |