Problem
Engagement-based models (collaborative filtering, user towers) work on item-level signals. They cannot say why a consumer buys organic produce or separate a dietary restriction from a casual preference, and their vectors are opaque to an LLM. The catalog lacks structured tags for semantics such as "plant-forward", and engagement history is sparse for new and single-vertical users. It is impractical for every surface to re-derive this from raw data.
Example from the article: a consumer who buys organic kale and almond milk likely prefers plant-forward food across categories not yet browsed, but an item-level model cannot infer it.
Solution in brief
An offline LLM pipeline turns behavior into versioned natural-language "memory blocks" per consumer (dietary preference, brand, taxonomy, stores). The blocks are encoded as dense embeddings (asymmetric encoding, then a two-tower projection) and as a consumer context graph. Both go to the feature store for ranking and retrieval, and the same text serves as LLM context.
Questions this case answers
- How to represent a user's durable preferences in a form that is both inspectable by language models and consumable by retrieval and ranking models when engagement history is sparse
- How to run LLM-based profile generation for a large user base on a batch budget while keeping outputs versioned and reversible
- How to turn raw behavioral signals into a consumer profile that both ranking models and LLMs can use?
- How to personalize for users with thin engagement history when the catalog lacks tags such as 'plant-forward'?
Tools and methods
- LLM memory generation — batch narratives and keywords grounded in the catalog
- Memory blocks and manifests — versioned components with lineage
- Pydantic — strict component schemas
- Asymmetric embeddings — consumer text with an instruction prefix, items without
- Two-tower model — projects embeddings to a lower-dimensional retrieval space
- Consumer context graph — consumers, brands, taxonomies and keywords as typed nodes
- Feature store — serves precomputed encodings at inference
Google or a one-time email code. No password.