Method

Inverse propensity scoring

IPS estimator reweighting logged feedback by action propensity.

8 cases from 8 companies apply Inverse propensity scoring. 5 were checked in an online A/B test.

Typical tasks: Evaluate models, Rank search results, Recommend to new users and new items, Measure impact without an A/B test and 1 more. Companies: Netflix, Avito, Sber, Lyft, Instacart, Thumbtack and 2 more. Often combined with: Doubly robust estimation, Off-policy evaluation, Gradient boosting, Multi-armed bandit, SHAP and 17 more.