How Transformers Increased Candidate Diversity on the Avito Home Page

Anatoly Mastryukov23 May 2026

Problem

Avito's Home page is an infinite feed with about 50% of all listing views. Its vector candidate generator first picks at most five category-region pairs per user with a counter-based heuristic, and only then does a transformer build one user vector per pair. The heuristic is a decay formula that nobody fully understands and that does not optimize CTR, so it caps the transformer. Dropping it for a single user vector collapses the history and makes nearest-neighbor search return one cluster.

Example from the article: a user viewed both apartments and cars. With one vector, the output is apartments of one type in one district.

Solution in brief

Replace the heuristic with a transformer that reads the user's click, favorite and contact history and predicts category-region pairs directly. Trainable CLS tokens give each target its own head, so one backbone serves several tasks and targets can be swapped in A/B tests.

Questions this case answers
  • How to replace a hand-tuned heuristic that selects segments for candidate generation with a learned model that predicts those segments directly
  • How to get diverse candidates from a single user representation when the user has several unrelated interests
  • How to choose which categories and regions to recommend from in a marketplace feed without a hand-tuned heuristic
  • How to raise the diversity and relevance of candidates from a vector candidate generator

Tools and methods

  • Transformer encoder (2 layers, dim 128) — reads up to 500 events of user history.
  • Trainable CLS tokens with per-feature classifiers — one distribution per target, trained together.
  • Held-out next-week target, softmax cross-entropy — one training scheme for all heads.
  • Recall@5 — offline metric, matching the five pairs production can serve.
  • PyTorch DDP, Polars — training on 8 GPUs.
Continue reading — it's free

Google or a one-time email code. No password.