Problem
Avito's home page is an endless feed of listings: about 50% of all listing views and about 30% of buyer-to-seller contacts. Its ranking has run on CatBoost since 2020. Quality saturated, since several times more data did not help. Strong embeddings had to be squeezed into dot products and counters, and every new signal needed hand-built features. The team wanted a neural ranker that uses embeddings and user history directly, without losing contacts or breaking production.
Example from the article: the first neural model, built by an AI agent with DCNv2 and mixture of experts, lost 3% NDCG offline and 6% users with contacts from recommendations to CatBoost. After five A/B tests the final model gained 4.3% on it.
Solution in brief
Put the neural ranker after CatBoost: CatBoost picks the top-300 of 5000+ candidates, and the neural model reorders them. Build it in small steps with an A/B test at each step: simple MLP, ListMLE loss, then a transformer over the last 50 user events and the candidates. Ship it blended by ranks with CatBoost.
Questions this case answers
- How to replace a gradient-boosted ranker with a neural ranker that uses user event history and embeddings, rolled out in small A/B-tested steps beside the existing model
- How to make offline ranking metrics predict online test results when re-ranking a large candidate set
- How to replace a gradient-boosted ranker with a neural ranker in a production recommendation feed
- How to use user history and embeddings in ranking when CatBoost cannot take them directly
Tools and methods
- ListMLE loss — listwise loss that picks the best remaining candidate at each step
- Quantile encoding of numeric features — 128 bins with interpolation
- Transformer over history and candidates — one pass, with an attention mask
- Rank blending with CatBoost — equal weights, no tuning
- CatBoost, ONNX Runtime, Triton — baseline and serving
Google or a one-time email code. No password.