Problem
About 10% of search queries on the Samokat Tech marketplace end with "nothing found". The usual causes are very short queries (a part number or an unknown brand) and very long ones (a full product description pasted from another site). The team wanted to recognize product attributes in the query and open the catalog page with filters already applied. The catalog has about 10 thousand fourth-level categories, each with its own filters, and no labeled data. Manual labeling is impractical: the assortment changes several times in half a year.
Example from the article: the query "iPhone 14" should lead to a page filtered to the values "iPhone 14 Pro" and "iPhone 14 Pro Max".
Solution in brief
Classify the query into a category first, which narrows 10 thousand filter sets to a few dozen candidate entities. Classification is nearest-neighbor search over embeddings of past queries, labeled with the categories of products users interacted with. Then tag tokens with a probabilistic model built from the catalog and map them to filter values with a per-category LSH index.
Questions this case answers
- How to extract structured attribute filters from free-text queries over a very large catalog without labeled data, when the catalog changes several times in half a year
- How to route a free-text query to the right one of thousands of categories by nearest-neighbour search over past queries, and map its tokens to filter values quickly
- How to extract product filters from search queries without manual labeling
- How to reduce the share of marketplace searches that return nothing
Tools and methods
- Catalog-based probabilistic labeling with beam search — token tags without annotators
- Sentence transformer or Word2Vec + ANN (HNSWlib) — classifies queries by neighbors' categories
- Per-category entity search and LSH (MinHash) — turns tokens into filter values
- Transliteration and synonym dictionaries — Cyrillic brands, unknown attributes
- W2NER-like model, LSTM, Russian BERT — compared as NER models
Google or a one-time email code. No password.
