An LLM Reads Marketplace Reviews for You

Maksim Bogdanov25 May 2024

Problem

A marketplace product can have 40 thousand or even 100 thousand reviews, so neither buyers nor sellers can read them all. The Wildberries Trust & Safety team wanted a short pros-and-cons summary per product card: buyers see the main points at once, conversion may grow, and sellers see recurring complaints. There was no labeled data, so nobody knew what a good summary looks like or what to train on.

Example from the article: for hair rollers, an extractive model plus a T5 generator produced the pros "convenient, hold well" and "very convenient" as separate points.

Solution in brief

Use an extractive model plus a T5 generator only to produce candidates, and let assessors pick the better one or write their own. Fine-tune Mistral-7B with LoRA on those labels, then tune it on pairwise preferences with IPO.

Questions this case answers
  • How to generate a pros-and-cons summary of many user texts with an LLM when no labeled reference summaries exist
  • How to bootstrap labels for a generative summarization task by using extractive output as a cheap annotation tool and then tuning on pairwise preferences
  • How to generate pros-and-cons summaries of product reviews with an LLM when there is no labeled data
  • How to bootstrap annotation for a generative summarization task using extractive models

Tools and methods

  • LexRank, BERT-extractive-summarizer — pick key sentences from reviews to get first candidates
  • SiberianFredT5-instructor — turns extracted sentences into a pros-and-cons list
  • Mistral-7B-Instruct-v0.2 with LoRA — the SFT model, best of 10–12 tried
  • ru-longformer-base-4096 — reward model on long inputs
  • IPO (a DPO loss variant) — preference tuning without a reward model
  • Side-by-side with five assessors, LLM-as-a-Judge — quality evaluation
Continue reading — it's free

Google or a one-time email code. No password.