From Scoring to Spelling: Rebuilding Ads Retrieval at Instacart

Karuna Ahuja3 Jun 2026

Problem

Instacart's ads retrieval used a BERT-style sequence model that treated each product ID as a token and scored every ID in its vocabulary against the session.

It hit three limits. A bigger vocabulary raised model size and latency, left rare items with sparse data, and never caught up with new products. The model memorized co-occurrences and favored frequent items over newer ones that fit the session. A flat distribution over all IDs also let off-topic items into the candidate set.

Example from the article: a barbecue cart (ground beef, buns, lettuce) got milk instead of an emerging brand's mustard, and a breakfast cart could pull in laundry detergent.

Solution in brief

Replace atomic product IDs with Semantic IDs: short codes from a residual-quantization autoencoder (RQ-VAE), where similar products share code prefixes. A generative model reads retailer type, user history and cart, and produces the Semantic ID of the next item. Beam search yields several code sequences, which a retailer-partitioned index maps to ad products.

Questions this case answers
  • How to retrieve candidates for a session-based recommender when the item vocabulary is too large and grows faster than the model can be retrained
  • How to make a retrieval model generalize to items with no interaction history instead of memorizing item co-occurrences
  • How to retrieve relevant sponsored products from a large catalog that keeps changing, including items with no purchase history?
  • How to stop a session-based retrieval model from returning off-topic items next to good ones?

Tools and methods

  • Semantic IDs (RQ-VAE) — four-token product codes
  • TIGER-style generative retrieval — an autoregressive model generates the next item's code
  • Beam search with temperature sampling — builds the candidate set and controls diversity
  • Retailer-partitioned index — maps generated codes to available ad products
  • TensorRT-LLM and NVIDIA Triton — GPU serving of the decoder
Continue reading — it's free

Google or a one-time email code. No password.