Your Opinion Matters

19 Jul 2022

Problem

Stitch Fix clients leave free-text notes and item reviews, hundreds of millions of them. Stylists must read a client's notes before every Fix, and a loyal client can have 100+ comments. Recommendation algorithms did not use this text, although the team's analysis showed that written feedback is essential for good personalization. Reading is slow for people, and language is hard for models.

Example from the article: a client writes "It all seems a bit too business casual for me. I'm looking for something a little bit more bohemian and casual." The system needs "dislikes business casual, likes bohemian, likes casual".

Solution in brief

Two fine-tuned GPT-3 models turn text into structure: one extracts product characteristics from item feedback, the other extracts each client's likes and dislikes from their notes. Stylists write the training labels. Each disliked characteristic is matched to the available inventory by a search query, and matching items get a lower probability of being shown to the stylist.

Questions this case answers
  • How to convert unstructured user feedback into structured preference signals when labeled data is scarce and only domain experts can label it
  • How to apply extracted negative preferences to a ranked candidate list while a human reviewer keeps the ability to override
  • How to turn free-text client notes and item reviews into structured likes and dislikes a recommender can use?
  • How to extract product attributes from review text when the client also mentions features the item lacks?

Tools and methods

  • GPT-3 fine-tuning — one model for product characteristics ("Opinio"), one for client likes and dislikes
  • Stylist labeling (experts in the loop) — about 10k training examples for each model
  • Search over item attributes — finds inventory items that match a client's dislike
  • Probability suppression in the recommender — lowers the chance of showing matched items; stylists can override
Continue reading — it's free

Google or a one-time email code. No password.