Deploying Transformers in Yandex Music

Daniil Burlakov27 May 2023

Problem

Yandex Music recommends tracks interactively in "My Wave": the backend asks ML for a batch of 5 tracks, and an active action such as a like triggers a new request. The online stack (candidate generators, then two CatBoost rankers) has to answer within the response time over a catalog of more than 100 million tracks, so it cannot run heavy models. It tracks the current context well but understands long-term interests worse. The team wanted a model that reads the long like history and lifts retention, listening time and diversity.

Example from the article: a user liked an artist a second ago, and the next batch must already reflect it.

Solution in brief

Train a two-tower model whose user tower is a six-layer transformer over the last 512 likes. Pre-train it with a contrastive loss on years of history, then fine-tune on recommendation sessions. Run it offline: embeddings refresh daily, the model retrains weekly. Online, the embeddings feed a candidate generator and CatBoost features.

Questions this case answers
  • How to add a long interaction history to a real-time candidate ranking stack under a tight latency budget by computing user representations offline
  • How to pre-train a transformer user encoder on years of interaction logs with a contrastive loss and then fine-tune it on a smaller set of session outcomes
  • How to use a long listening or like history in a music recommender
  • How to build and train a transformer user tower for recommendations

Tools and methods

  • Transformer user tower — encodes the like history into three user embeddings
  • Two-tower model — small item tower, relevance is a dot product
  • Pointwise, pairwise and NT-Xent-like losses — one user head per loss
  • Pre-train and fine-tune — contrastive pre-training, then training in the recommendation context
  • CatBoost — light and heavy ranking stages with embedding features
  • Embedding quantization, nearest-neighbor search — cheap serving
Continue reading — it's free

Google or a one-time email code. No password.