Cold-Starting Sales of Complementary Products with LLMs

Artem Zemlyak24 May 2025

Problem

Samokat, a quick-delivery service built on ecom.tech technology, added a Quick Non Food (QNF) assortment of hundreds of thousands of new items. Users had to learn these products exist, and search results were the place to show them as complements to the query. The team had no statistics on the items, so no baseline and no training data. The whole pipeline had to launch from scratch in about a month and a half, and random items were not acceptable.

Example from the article: for the query "bread" the system should surface a toaster or a cutting board (interest 5 of 5 for experts), not wooden toothpicks (1 of 5).

Solution in brief

Instead of a model trained on behavior, the team used labeling. An open-source LLM (Qwen2.5-72B) labels query-product pairs by Exact, Substitute, Complement and Irrelevant (ESCI) classes and keeps the complements. Interest ranking uses LLM-generated scores as features of a small ML model trained on one round of expert ratings, followed by a randomizing heuristic.

Questions this case answers
  • How to recommend complementary items for a new catalog segment with no interaction history by using generated relevance labels instead of behavioural data
  • How to rank candidate items when there are no clicks or purchases to train on and only a few expert ratings exist
  • How to recommend complementary products for a new catalog segment with no interaction history
  • How to label query-product relevance or complementarity cheaply with an open-source LLM instead of human labelers

Tools and methods

  • ESCI labeling by an LLM — selects complementary categories and products
  • Qwen2.5-72B-Instruct (AWQ, FP8) on SGLang — labeling at scale on A100 or H200
  • Rerankers and embedders (bge-reranker-v2-m3 and others) — cheap first filter for confident positives
  • LLM scores plus lasso, ridge, LightGBM — interest model per gender
  • Group-and-sample heuristic — fights the feedback loop
Continue reading — it's free

Google or a one-time email code. No password.