Problem
Mercari is a Japanese customer-to-customer marketplace. Sellers describe listings in their own words, what matters differs by category, and the wording keeps changing. The team wants attributes such as size, color and condition pulled out of that text to understand listings and help sellers improve them. Classic extractors need retraining when wording shifts. Commercial LLM APIs handle varied text, but at production volume they stay too expensive and hallucinations are hard to control by prompting.
Example from the article: for the listing "A Mercari T-shirt size M, blue. Used only once and kept in a clean wardrobe after." and the keys size, color, original retail price, the model returns Size: M, Color: Blue, Original retail price: NONE.
Solution in brief
Fine-tune a small open model (gemma-2b-it) with QLoRA. The prompt holds the description and the list of attribute keys, and the model answers "key: value" lines with NONE for absent values. A 4-bit GGUF quantization shrinks it for serving. On BLEU it beat gpt-3.5-turbo-0125.
Questions this case answers
- How to extract a caller-specified list of fields from noisy user-written text when a commercial LLM API is too expensive at production request volume
- How to cut the serving cost of an LLM for a narrow task while keeping quality above a large commercial baseline
- How to extract attributes that differ by category from free-text listings without a commercial LLM API?
- How to make a small open LLM beat GPT-3.5 Turbo on a narrow extraction task?
Tools and methods
- QLoRA — train small adapters on a 4-bit base model on one GPU
- gemma-2b-it — 2-billion-parameter base model
- Hugging Face Transformers, PEFT, TRL, bitsandbytes — loading, adapters, SFT training, NF4 quantization
- Weights & Biases — versioned data split and run logs
- llama.cpp — GGUF quantization (q4_k_m) and inference
- GPT-3.5 Turbo — baseline
Google or a one-time email code. No password.
