Shaping product understanding with contrastive reinforcement learning

Pat Geitner, Drake Weissman26 May 2026

Problem

A marketplace of one-of-a-kind items cannot define a fixed attribute schema that captures what sets each product apart. The distinguishing details sit in images, nested attributes and long descriptions mixed with shipping and care text. Search and recommendation models cannot ingest all of this under tight latency limits, and not every system reads images. The quality of the product representation caps how well the models can match buyer taste to products.

Example from the article: a buyer searches "Sculptural Stoneware Pottery" and engages with a handbuilt stoneware bowl but not with a speckled pasta bowl set. The summary should show why the first listing fits.

Solution in brief

Fine-tune a vision-language LLM with GRPO to write short natural-language product summaries. The reward comes from a frozen search retriever and compares how close the summaries of an engaged and a non-engaged listing are to the query. Summaries that separate the two get a higher reward, so the model learns to surface the details that buyers respond to.

Questions this case answers
  • How to learn what a generated text representation of an item should contain when no fixed schema or prompt can say which details matter
  • How to fine-tune a generative model with a frozen downstream model as the reward without rewarding generic, keyword-stuffed outputs
  • How to turn long, noisy listing data (title, images, description, attributes) into a short text that search and recommendation models can use?
  • How to define what a good free-form product summary is when the important details differ for every product?

Tools and methods

  • Qwen3-VL-8B — open-source vision-language LLM that generates the summaries
  • Group Relative Policy Optimization (GRPO) — RL algorithm that updates the LLM from the relative quality of candidate summaries
  • Two-tower dense retriever (frozen) — production search model used only as the reward scorer
  • Triplet data (query, engaged listing, non-engaged listing) — contrastive training signal from search logs
Continue reading — it's free

Google or a one-time email code. No password.