Problem
A marketplace that grew from restaurants into grocery, convenience, alcohol, retail, flowers and gifting must help customers discover items across a catalog of hundreds of thousands of SKUs. A single relevance score does not say whether to show a favorite, a good deal or something new. Prompting an LLM with the whole catalog is impractical at this scale. The post does not quantify the cost of poor balance.
Example from the article: a user who orders ramen weekly gets instant ramen kits and Asian condiments in the next grocery session.
Solution in brief
Every retrieval, ranking and presentation decision is framed around three user values: familiarity, affordability and novelty. A classic recommender (two-tower retrieval, multi-task rankers) does the work at scale. LLMs add semantic understanding where text, cold start and long-tail intent matter, and hierarchical RAG with semantic IDs keeps their context small.
Questions this case answers
- How to balance familiarity, price fit and novelty in recommendations over a catalog too large to fit into an LLM prompt
- How to personalize a new product vertical for users whose history exists only in an adjacent vertical
- How to balance what a shopper already buys, what fits their budget and what is new in one retail discovery feed?
- How to add LLMs to a retrieval-and-ranking recommender without putting the whole catalog into prompts?
Tools and methods
- Two-tower embedding model — top-N retrieval of familiar items by dot product
- Multi-task rankers with mixture-of-experts — click, add-to-cart, in-session and delayed conversion per surface
- Value-to-Consumer objective — uprank items with the best value for a customer's price sensitivity
- Knowledge graphs and consumer clusters — cross-vertical novelty from restaurant history
- Hierarchical RAG — narrow the catalog by category tree before an LLM call
- Semantic IDs — compact embeddings that encode catalog hierarchy
Google or a one-time email code. No password.