Problem
Yandex.Eats ranks up to several thousand restaurants per user in the main feed, where most orders happen. The candidate set depends on the user's location and time, so it cannot be pre-ranked offline. Each new feature needs data, a dataset, retraining, an offline check and an A/B test, and this cycle gets slower as ideas get more complex. Some needs do not fit the model: a fraud penalty of a known size, a priority for nearby restaurants in a blizzard within hours, and features that promote restaurants users rarely saw before.
Example from the article: a restaurant with fraud should drop in the feed by a controlled amount, which changing the training target does not guarantee.
Solution in brief
The final score is a linear formula: the complex personalized model becomes one feature with the largest weight, next to interpretable features with hand-set weights. Every weight is expressed in minutes of delivery time, so it has a concrete meaning. Weights are edited in a web interface, with no retraining or deployment.
Questions this case answers
- How to apply a business rule or penalty of controlled size on top of a learned ranking model without retraining it
- How to test a new ranking signal when the logged data contains almost no outcomes for the positions it would promote items to
- How to ship and test a new ranking feature faster than the full retrain, offline check and A/B cycle allows?
- How to demote restaurants for fraud by an exact, controllable amount instead of hoping a retrained model learns it?
Tools and methods
- Two-stage ranking with re-ranking — the model scores a limited candidate set, the rest follow simple logic
- Gradient boosting — the personalized model predicting where the user will order
- Linear scoring layer over the model — the personalized score plus interpretable features with editable weights
- Admin panel (web interface) — changing weights and launching experiments without deployment
Google or a one-time email code. No password.
