Problem
DoorDash grocery search showed different results for queries with the same intent, such as «grape» and «grapes». Customers who typed the other form saw fewer options. Elasticsearch already normalizes both words to one stem during retrieval, so the same items are retrieved. The difference appears afterwards: the search service receives documents with the original fields and computes lexical text features for ranking and filtering on unnormalized text. Different word forms are treated as different words, which makes the features sparse and lowers relevance.
Example from the article: a search for «grapes» showed noticeably fewer options than a search for «grape».
Solution in brief
Move normalization out of Elasticsearch into the search service. Apply the same Lucene-based normalization to the query before retrieval and to item fields when catalog data is indexed, so text features are computed on normalized text. A second, smaller project uses string normalization plus DBSCAN clustering to pick popular taxonomy candidates from noisy menu data.
Questions this case answers
- How to make a search ranker treat different surface forms of the same query as one intent when the retrieval engine already normalizes but the ranking stage sees raw text
- How to surface the most frequent new entries from large volumes of noisy, near-duplicate short strings for manual review
- How to make search return the same results for query variants like «grape» and «grapes»?
- How to normalize queries and item fields so that ranking text features are not sparse?
Tools and methods
- Query and index normalization — same normalization of query and item name/description before feature computation
- Elasticsearch, Apache Lucene analyzers — retrieval and the source of the normalization methods
- DBSCAN with Levenshtein distance — groups near-duplicate strings; cluster size shows popularity
- Apache Airflow — daily ETL for the clustering job
- A/B experiment — measures null search rate, click-through, add-to-cart and perfect search rates
Google or a one-time email code. No password.