Problem
Retail media teams show sponsored products next to organic search results. Performance models optimize clicks, but an irrelevant ad annoys shoppers, lowers the perceived quality of the retailer's search and hurts trust in the ad platform. Relevance must also hold across languages, regions and verticals, where one word can mean different things. Labeling query-product pairs by hand does not scale, and raw clicks are biased by position and popularity.
Example from the article: for the query "red dress", other brands, styles or sizes of red dresses are acceptable, but a yellow dress, jeans or handbags are not.
Solution in brief
Train a two-tower model (a keyword encoder and a product encoder) with contrastive learning on keyword-product pairs rebuilt from search and product-view logs and filtered hard. Evaluate it on human-labeled pairs. Use its similarity score as a relevance threshold before performance ranking.
Questions this case answers
- How to train a query-to-item relevance model from biased, indirect click logs when labeling pairs by hand is impossible at scale
- How to enforce a minimum semantic relevance threshold in a latency-bound ranking pipeline that scores billions of pairs a day
- How to score how well a search keyword matches a product when you have no labeled query-product pairs at scale?
- How to train a relevance model on click logs without learning position and popularity bias?
Tools and methods
- Two-tower model — separate keyword and product encoders; relevance is the similarity of their embeddings
- Contrastive learning — pulls matched keyword-product pairs together and pushes mismatched pairs in the batch apart
- Click-log pair reconstruction with quality filters — builds positive training pairs without manual labels
- Human-labeled evaluation set — ground truth for semantic relevance
- Ray with Distributed Data Parallel — multi-GPU training on billions of pairs
- CLIP — tried first, a poor fit for short keywords
Google or a one-time email code. No password.