Problem
An online store with over 400 million SKUs shows items inside a category tree (Toys → Outdoor Toys → Pool Toys). Items are sometimes miscategorized, so a category page can show irrelevant products. A second, hidden hierarchy, the product type, says what an item is for: a dentist's screwdriver is "Oral Care Accessories", an electronics one is "Screwdriver Tool". For every category the team needs the product types that truly belong there, to filter out other items. Click data is a poor signal, because customers click by mistake or out of curiosity. The candidate space is millions of category and product-type pairs.
Example from the article: in the "Pool Toys" subcategory the valid product types include "Water Slides", "Pool Floats" and "Water Blasters".
Solution in brief
Ghotok scores category and product-type pairs in two stages. Trained predictive models, tuned on a small human-labeled set, cut the pairs from millions to thousands at a fixed false positive rate. A generative model with a chain-of-thought prompt then removes the remaining irrelevant pairs. The result is stored offline and served from a cache.
Questions this case answers
- How to decide which labels from one hierarchy are valid for each node of another hierarchy when behavioral signals are noisy and there are millions of candidate pairs
- How to combine a cheap trained classifier with an expensive LLM judge so that the LLM sees only a small share of the candidates
- How to find which product types belong in each node of a category tree without click data?
- How to keep wrongly categorized items out of a category page?
Tools and methods
- Predictive model ensemble — scores each pair from domain-specific features, trained on human labels
- Fixed-FPR confidence thresholds — drop pairs below a learned cutoff
- Generative model with chain-of-thought prompt — second-stage relevance judge without training
- Two-tier LRU cache — serves product-type sets in milliseconds
Google or a one-time email code. No password.