X-split ARGUS: An Autoregressive Encoder-Decoder for Ad Ranking

Alexander Ploshkin23 May 2026

Problem

Yandex ranks banners in its ad network and in search ads, where little is known about each user. A transformer over the user's event history ranks well only if the history is long. The two-tower setup squeezes the history into one vector, which caps quality. Letting candidates attend to the history (early fusion) lifts quality, but a transformer over about 8000 events is too slow at runtime. Production could afford only about 200 events.

Example from the article: on 8192 events, early-fusion Target-Aware ARGUS gave +2.1% log-likelihood gain (llp) against +1.1% for the two-tower model, but could not be deployed.

Solution in brief

X-split ARGUS splits the history in two. An offline encoder reads the long history (8192 events) once a day and leaves 8 embeddings per user. A runtime encoder reads the latest 256 events. A small cross-attention layer lets about 300 candidates look at both. Distillation, smoothed negatives and chronological training bring the offline gain to +1.8% llp.

Questions this case answers
  • How to rank candidate items for a user from a long interaction history when serving latency allows only a short window of the most recent events
  • How to combine long-history user signals with candidate-aware ranking when the expensive history encoder cannot run once per candidate at request time
  • How to rank ads with a sequential transformer model that uses long user history under strict runtime latency limits
  • How to move from a two-tower (late fusion) model to candidate-aware (early fusion) ranking without recomputing a long-history transformer per candidate

Tools and methods

  • ARGUS — autoregressive transformer over tokenized user events, with a context token per hit
  • Cross-attention layer — early fusion of candidates with the compressed history
  • Windowed attention — limits the runtime encoder's context during one-pass training
  • Logit distillation (L1) — Target-Aware ARGUS as teacher, training-only
  • Chronological training — time chunks with target decay
Continue reading — it's free

Google or a one-time email code. No password.