Method

Off-policy evaluation

Estimating a policy value from logged data (IPS, SNIPS, direct method).

8 cases from 7 companies apply Off-policy evaluation. 4 were checked in an online A/B test.

Typical tasks: Evaluate models, Decide who gets an offer, Optimize decisions, Measure impact without an A/B test and 4 more. Companies: Lyft, Yandex Lavka, Sber, Instacart, Spotify, Pinterest and 1 more. Often combined with: Inverse propensity scoring, Doubly robust estimation, CATE / HTE estimation, Multi-armed bandit, Uplift evaluation (Qini/AUUC) and 16 more.