Problem
A ride-hailing marketplace must decide how to split budget between rider prices and driver incentives. Long-term effects of these decisions tend to outweigh the short-term ones: charging riders more earns profit today but loses riders and drivers later. Part of the long-term effect is direct (an incentivized driver returns more often). The harder part is market-mediated: more driver hours cut rider wait and surge, but also leave drivers idle and less willing to return. No single experiment verifies it.
Example from the article: raise driver incentive spend by in week 1, then estimate how rider return rates and future driver hours change through shorter waits and higher idleness.
Solution in brief
Assume market-mediated effects pass fully through a few negative user experiences (long waits, high surge, cancellations, idleness). Step 1 estimates how a policy shifts those experiences. Step 2 estimates how they change future behavior. Direct effects are added, and the total is checked in region-split experiments.
Questions this case answers
- How to estimate the long-term effect of a policy change when the effect passes through other participants of a two-sided market and a long randomized test is too slow
- How to validate a chain of observational causal estimates when each link can be checked only by a different imperfect experiment
- How to estimate the long-term effect of a price or incentive change in a two-sided marketplace without running a long experiment?
- How to separate direct long-term effects from effects that pass through the market (wait times, surge, driver idleness)?
Tools and methods
- Residualized regression — policy effect on negative experiences, measured as deviation from the time-of-week norm
- AIPW (doubly robust estimator) — effect of negative experiences on future rides, retention and driver hours
- Surrogacy index — scales a short-term experience shift into a long-term effect
- Switch-back, user-split and region-split experiments — validate each step and the final forecast
Google or a one-time email code. No password.