Problem
Instacart's ads retrieval used a BERT-style sequence model that treated each product ID as a token and scored every ID in its vocabulary against the session.
It hit three limits. A bigger vocabulary raised model size and latency, left rare items with sparse data, and never caught up with new products. The model memorized co-occurrences and favored frequent items over newer ones that fit the session. A flat distribution over all IDs also let off-topic items into the candidate set.
Example from the article: a barbecue cart (ground beef, buns, lettuce) got milk instead of an emerging brand's mustard, and a breakfast cart could pull in laundry detergent.
Solution in brief
Replace atomic product IDs with Semantic IDs: short codes from a residual-quantization autoencoder (RQ-VAE), where similar products share code prefixes. A generative model reads retailer type, user history and cart, and produces the Semantic ID of the next item. Beam search yields several code sequences, which a retailer-partitioned index maps to ad products.
Questions this case answers
- How to retrieve candidates for a session-based recommender when the item vocabulary is too large and grows faster than the model can be retrained
- How to make a retrieval model generalize to items with no interaction history instead of memorizing item co-occurrences
- How to retrieve relevant sponsored products from a large catalog that keeps changing, including items with no purchase history?
- How to stop a session-based retrieval model from returning off-topic items next to good ones?
Tools and methods
- Semantic IDs (RQ-VAE) — four-token product codes
- TIGER-style generative retrieval — an autoregressive model generates the next item's code
- Beam search with temperature sampling — builds the candidate set and controls diversity
- Retailer-partitioned index — maps generated codes to available ad products
- TensorRT-LLM and NVIDIA Triton — GPU serving of the decoder
Google or a one-time email code. No password.