Problem
Yandex Music recommends tracks interactively in "My Wave": the backend asks ML for a batch of 5 tracks, and an active action such as a like triggers a new request. The online stack (candidate generators, then two CatBoost rankers) has to answer within the response time over a catalog of more than 100 million tracks, so it cannot run heavy models. It tracks the current context well but understands long-term interests worse. The team wanted a model that reads the long like history and lifts retention, listening time and diversity.
Example from the article: a user liked an artist a second ago, and the next batch must already reflect it.
Solution in brief
Train a two-tower model whose user tower is a six-layer transformer over the last 512 likes. Pre-train it with a contrastive loss on years of history, then fine-tune on recommendation sessions. Run it offline: embeddings refresh daily, the model retrains weekly. Online, the embeddings feed a candidate generator and CatBoost features.
Questions this case answers
- How to add a long interaction history to a real-time candidate ranking stack under a tight latency budget by computing user representations offline
- How to pre-train a transformer user encoder on years of interaction logs with a contrastive loss and then fine-tune it on a smaller set of session outcomes
- How to use a long listening or like history in a music recommender
- How to build and train a transformer user tower for recommendations
Tools and methods
- Transformer user tower — encodes the like history into three user embeddings
- Two-tower model — small item tower, relevance is a dot product
- Pointwise, pairwise and NT-Xent-like losses — one user head per loss
- Pre-train and fine-tune — contrastive pre-training, then training in the recommendation context
- CatBoost — light and heavy ranking stages with embedding features
- Embedding quantization, nearest-neighbor search — cheap serving
Google or a one-time email code. No password.
