Problem
Netflix runs many specialized recommendation models, for example "Continue Watching" and "Today's Top Picks for You". Each is trained on its own, so maintenance is costly and gains in one model do not carry over to the others. Most models see only a short recent slice of a member's history, because serving latency and training cost cap the window. Each new title also needs a preference estimate before anyone has watched it.
Example from the article: a newly launched or pre-launch title needs a reasonable embedding with no member interactions at all.
Solution in brief
Train one large sequential model on the full interaction history with a GPT-style next-token objective. Tokens are merged interactions with context features, and each title is embedded as a mix of a learned ID embedding and metadata embeddings, weighted by title age. Downstream models take its predictions, its stored embeddings, or fine-tune it.
Questions this case answers
- How to centralize preference learning in one pretrained sequence model that many downstream ranking models can reuse
- How to train a sequence model on user histories far longer than the context window that serving latency allows
- How to represent items that appear continuously, so that a new item with no interactions still gets a usable embedding
- How to replace many separately trained recommendation models with one model that learns member preferences once?
Tools and methods
- Sequential recommendation with next-token prediction — pretrain on unlabeled interaction sequences to learn long-term preferences.
- Interaction tokenization — merge actions on the same title into one token to shorten sequences.
- Multi-token and auxiliary-target prediction — predict several next items and sequences such as genre to reduce myopia and overfitting.
- Sliding window sampling and sparse attention — cover long histories during training.
- KV cache — reuse computation in multi-step decoding.
- Metadata plus ID title embeddings — give new titles usable embeddings.
Google or a one-time email code. No password.