Meta Adaptive Ranking Model: Bending the Inference Scaling Curve to Serve LLM-Scale Models for Ads

Xi Chen, Yuxin Chen, Zeliang Chen, Jonathan Herbach, Jian Jiao, Jinfu Leng, Dianshi Li, Gaoxiang Liu, Liang Luo, GP Musumeci, Michael Shao, Ellie Wen, Ruichao Xiao, Yavuz Yetim, Nancy Yu, Zhengkai Zhang31 Mar 2026

Problem

Ads ranking must pick and return ads in under a second for billions of people. Meta wanted its ads runtime model at LLM-scale complexity to read user intent better. A bigger model needs more compute and memory, which breaks the latency limit or the cost budget. Adding hardware does not pay back.

Example from the article: a model of O(10 GFLOPs) per token, comparable to top LLMs, had to run with latency bounded at O(100 ms).

Solution in brief

Three layers cut the cost. First, compute user-side signals once per request and share them across ad candidates. Second, shape the model and kernels to the hardware, with selective FP8 and fused operators. Third, shard huge embedding tables across GPUs. A router sends complex user journeys to the heavy model.

Questions this case answers
  • How to scale a ranking model's complexity by an order of magnitude while keeping per-request latency and serving cost flat
  • How to remove redundant computation when one request is scored against many candidates
  • How to serve a much heavier ads ranking model without breaking a sub-second latency budget?
  • How to avoid paying for the same user computation once per ad candidate?

Tools and methods

  • Request-oriented computation sharing — compute heavy user signals and long behavior sequences once per request, not per candidate
  • In-kernel broadcast — share request-level embeddings across candidates inside the GPU kernel
  • Wukong Turbo — runtime version of the Wukong stacked factorization machine architecture
  • Selective FP8 post-training quantization — low precision only in layers that tolerate it
  • Graph and kernel specialization — operator fusion, Grouped GEMM, horizontal fusion
  • Multi-card embedding sharding — split terabyte-scale tables across GPUs
  • Adaptive compute routing — send complex user journeys to the LLM-scale model
Continue reading — it's free

Google or a one-time email code. No password.