Problem
Stitch Fix launched Freestyle, where algorithmically built outfits go straight into each client's shopping feed, around 4 million clients. In the earlier Fix product, expert stylists reviewed every selection first. Without that review the team had no direct way to tell whether the outfits were good. Engagement and revenue are indirect signals. There was also no ready dataset describing what a good outfit is for a given client.
Example from the article: outfits for the "Trending for You" and "Complete Your Looks" shops in Freestyle.
Solution in brief
Stylists play two roles. First, they grade a daily random sample of shown outfits against a rubric (Outfit QA), which gives a quality measure, a map of recurring mistakes and labels for a model that predicts outfit problems. Second, they create outfits in a tool built for them, which gives a continually refreshed training set and extra content to show clients.
Questions this case answers
- How to measure the quality of recommendations shown without human review when engagement and revenue are only indirect signals
- How to bootstrap training data for a new recommendation product from in-house domain experts and keep it fresh as the catalog changes
- How to judge whether algorithmically generated outfits are good when no expert reviews them before the customer sees them?
- How to get training data for a personalized outfit recommender when no ready dataset exists?
Tools and methods
- Outfit QA — daily expert grading of a random sample of shown outfits with a labeling tool and a rubric
- Loss-pattern analysis — grouping QA problems by category to prioritize model fixes
- Outfit QA problem predictor — a model trained on QA labels to estimate the chance that an outfit has a problem before it is shown
- Stylist-generated outfits — experts build outfits from sampled inventory; used as training data and shown to clients
Google or a one-time email code. No password.