Method

Difference-in-differences

Effect estimate by comparing pre/post change of treated and control groups.

13 cases from 9 companies apply Difference-in-differences. 1 was checked in an online A/B test; 7 report results in production.

Typical tasks: Measure impact without an A/B test, Get experiment results faster, Experiment when users affect each other, Avoid false wins in A/B tests and 4 more. Companies: DoorDash, Instacart, Walmart, Avito, Spotify, Nextdoor and 3 more. Often combined with: Nearest-neighbor matching, Synthetic control, Power analysis / MDE, Switchback test, Holdout / global control group and 35 more.