GenRecs Tokenization: Transforming Ads Recommendation at Meta with Stable Model Vocabulary

Carl Hu, Justin Johnson, Parish Aggarwal, Kaushik Rangadurai, Senthil Manickavelu, Tianwei Zhang19 Oct 2026

Problem

Ads ranking and retrieval models learn from per-item IDs, and every new ad, product or page starts with an empty representation. Tail ads with few impressions never accumulate enough signal, so the model ranks them poorly and they win few auctions. That cuts the training data for exactly the items that need it.

Example from the article: a sportswear brand launches next season's trail shoe, and the model should reuse what it learned from last season's shoe before anyone has seen the new ad.

Solution in brief

Replace the dedicated ID of each item with a short sequence of tokens from a small, stable semantic vocabulary. Each item is encoded from its content and engagement signal, split into subspaces and quantized against codebooks. Similar items end up sharing tokens. Tokenizer versions are then compared on cheap intrinsic metrics that track downstream CTR accuracy.

Questions this case answers
  • How to represent items that have no interaction history so that a ranking model can reuse what it learned about similar items
  • How to choose between candidate item encoders cheaply, before running full downstream model training
  • How to give new and low-impression ads a useful representation when they have no engagement history?
  • How to carry what a ranking model learned about last season's items over to this season's items?

Tools and methods

  • Semantic tokenization (semantic IDs) — represent an item by several discrete tokens instead of one ID.
  • Pre-trained multimodal encoder — turns the text and images of an ad, product or page into an embedding with world knowledge.
  • Residual decomposition and learned vector codebooks — split the embedding into parts and map each to a codebook entry.
  • Intrinsic tokenizer metrics — cluster purity, distribution balance and codebook utilization, used to pick tokenizer versions.
Continue reading — it's free

Google or a one-time email code. No password.