Problem
Etsy lists more than 100 million items from more than 5 million third-party sellers. Many items are handmade or custom, so there are no global SKUs to map to attributes. Attribute fields exist but are mostly optional, so sellers put key facts such as dimensions into titles, descriptions or photos. Search filters, product comparison and ML models need structured attributes, and reading long text or images at query time is too slow for search.
Example from the article: a listing that looks like a t-shirt is really a porcelain sculpture, and only the seller's text says so.
Solution in brief
An LLM pipeline reads each listing's title, description and images and returns structured attributes. Quality is measured against reference labels, which a strong LLM generates and domain experts review ("silver labels") instead of relying only on human annotation. A cheaper, scalable LLM is tuned against these labels, and inference is guarded by validation, error alerts and a model-quality check before each run.
Questions this case answers
- How to convert unstructured item descriptions into structured fields across millions of items and thousands of categories with no global product identifiers
- How to build and maintain an evaluation set for LLM extraction when human annotators are costly and make mistakes
- How to turn free-text listings and photos from third-party sellers into structured product attributes at marketplace scale?
- How to evaluate LLM attribute extraction when human-labeled ground truth is slow, expensive and error-prone?
Tools and methods
- LLM attribute extraction — one prompt per attribute, run in parallel
- Few-shot examples and taxonomy rules — expert-chosen examples and category rules go into the context
- Silver labels — reference labels from a high-quality LLM, reviewed by Etsy domain experts
- LiteLLM — routes LLM requests across regions
- Pydantic — parses responses into dataclasses with type and business-logic validation
Google or a one-time email code. No password.