Method
Off-policy evaluation
Estimating a policy value from logged data (IPS, SNIPS, direct method).
8 cases from 7 companies apply Off-policy evaluation. 4 were checked in an online A/B test.
Typical tasks: Evaluate models, Decide who gets an offer, Optimize decisions, Measure impact without an A/B test and 4 more. Companies: Lyft, Yandex Lavka, Sber, Instacart, Spotify, Pinterest and 1 more. Often combined with: Inverse propensity scoring, Doubly robust estimation, CATE / HTE estimation, Multi-armed bandit, Uplift evaluation (Qini/AUUC) and 16 more.
How We Solve Yandex Lavka's Optimization Problems with Uplift Models![]() | ![]() | food-delivery | 2026-05 |
| The New Policy Hasn't Run Yet: Comparing It to the Old One Before an A/B Test | banking | 2026-05 | |
| Lyft’s Reinforcement Learning Platform | mobility | 2024-03 | |
| Using Contextual Bandit models in large action spaces at Instacart | e-commerce | 2023-06 | |
| Experimenting with Machine Learning to Target In-App Messaging | media-streaming | 2023-06 | |
| Inverse Propensity Score based offline estimator for deterministic ranking lists using position bias | food-delivery | 2022-08 | |
| Pricing at Lyft | mobility | 2022-08 | |
| Experiment without the wait: Speeding up the iteration cycle with Offline Replay Experimentation | social | 2022-01 |
