Problem
An ads ranking system scores thousands of candidate ads within milliseconds. A transformer over a long user history improves relevance. The common hybrid setup runs it next to a separate model for sparse features. That setup loses information between the two models, still needs hand-built features, and the two parts interfere as they grow. Longer sequences and deeper transformers then hit the serving budget.
Example from the article: Meta's ads ranking for Facebook and Instagram, where billions of daily interactions feed the model.
Solution in brief
Split the sequence model in two. A deep transformer runs asynchronously on each user's long history and caches user embeddings. A light online ranker combines those cached embeddings with fresh signals and the ad being scored. Capacity then grows offline, while the online path stays within its latency limit.
Questions this case answers
- How to feed long behavioral histories into a real-time ranking model when per-request compute is capped by a strict latency budget
- How to increase a ranking model's capacity predictably while keeping the serving path almost unchanged
- How to use long user histories (thousands of events) in a ranking model without breaking the latency budget?
- How to scale a sequence model's capacity without a matching rise in serving cost?
Tools and methods
- Offline user model — deep transformer over event sequences thousands long, output cached per user
- Online ranking model — fast model that reads cached embeddings plus fresh user and ad signals
- Dense tokenization — puts sparse features and behavior events into one dense vocabulary so attention learns interactions
- Target-aware multi-head attention — weighs past behaviors against the ad being scored
- Scaling-law analysis — tracks normalized entropy (NE) against compute across depth, width, sequence length and content features
Google or a one-time email code. No password.