Experience with Transformer-Based Personalization at Avito

Aleksandr Mikheev24 May 2025

Problem

Avito recommends ads through candidate generators, a ranker and a final feed. The main neural generator, user2vec, builds a user embedding as a time-decayed weighted sum of the embeddings of ads in the history. It is fast, but it assumes a linear dependence between history and the next ad, ignores action order and does not use the many search queries users make. The model must also work in real time.

Example from the article: a user views ads for animals. The recommender should follow this interest and still use longer-term ones.

Solution in brief

Train a two-tower model. The ad tower produces content-based embeddings that are cached, and the user tower is a transformer over the sequence of ad actions and search queries. Training samples include a random time gap before the target ad and drop the actions that led to it. The model serves both as a candidate generator and as a feature for the ranker.

Questions this case answers
  • How to replace averaged item embeddings with a sequence model over a user's action history and queries so that item vectors can still be precomputed and indexed for real-time candidate retrieval
  • How to build training samples for a two-tower retrieval model so that it does not simply memorize items similar to the recent history
  • How to replace averaged item embeddings with a transformer user tower for real-time personalized candidate retrieval
  • How to account for the order of user actions and search queries in a user embedding

Tools and methods

  • Two-tower model — dot product of a user embedding and an ad embedding
  • Transformer user tower — encodes the action and search sequence; the CLS output is the user embedding
  • Deep Cross Network (DCN) — feature interactions inside the ad and search encoders
  • InfoNCE / SigLIP loss — in-batch and sampled negatives
  • Redis and an ANN index in Sphinx — cached ad vectors and nearest neighbor search
Continue reading — it's free

Google or a one-time email code. No password.