Problem
Netflix's production ranker relies on thousands of hand-crafted features and specialized architectures. Adding a content type or a product surface needs new features, architecture changes, infrastructure work and experiments. Off-the-shelf LLMs do not fix this: they over-recommend globally popular titles, invent items outside the catalog, ignore business constraints and personalize weakly. The team wanted a ranker that learns from raw text histories and still beats a ranker tuned over many years.
Illustrative example: a member's plays, thumbs-up and abandons are written out as text, and the model ranks the whole catalog for the current surface and device.
Solution in brief
GenRec post-trains an internal foundation LLM on conversations built from interaction logs. A scoring head over learned item embeddings replaces text generation, so output is always an in-catalog ranking. Reward models weight the ranking loss toward long-term value. The prompt is trimmed to a token budget, and inference is one prefill pass.
Questions this case answers
- How to replace a feature-heavy ranking model with a language-model ranker while keeping it restricted to a fixed item catalog
- How to cut the inference cost of a text-input ranker when history length and candidate count are large
- How to build a recommendation ranker on top of an LLM from text-verbalized user histories?
- How to keep an LLM ranker inside the catalog and make it score thousands of items cheaply?
Tools and methods
- Context engineering — builds a compact text prompt from logs within a fixed token budget
- Supervised fine-tuning — Phase 1 adapts an open LLM, Phase 2 trains the ranker
- Reward model — weights each training example by long-term value
- Sampled softmax — keeps training tractable for large catalogs
- Prefix caching — reuses shared prompt prefixes to cut serving cost
- Knowledge distillation — smaller backbones at lower cost
- vLLM — serving stack, run in prefill-only mode
Google or a one-time email code. No password.