Reviews in Organization Search on Yandex Maps

Andrey Danilchenko4 Jun 2022

Problem

Yandex Maps geosearch retrieves organizations by categories and features extracted from the query. This model cannot express fuzzy intent: "cozy cafe" has no matching feature. Results also match formally but not by meaning. A year before the talk, the query "sushi SPB" returned a strip club because it serves sushi. The team wanted to cut such irrelevant results. Success is judged by expert annotators who rate how well each organization fits the query.

Example from the article: the query "sushi SPB" showed a strip club; after the change only real sushi bars remain.

Solution in brief

A search-tuned BERT is fine-tuned on a small set of expert relevance labels for complex queries. It is then distilled into a fast DSSM on year-long logs after fixing the query bias in that dataset. The DSSM works as a filter in the post-filtering stage, where irrelevant organizations are removed, not just ranked lower.

Questions this case answers
  • How to remove results that satisfy a query formally but not semantically when structured features cannot express the user's intent
  • How to deploy a heavy text relevance model in a latency-bound ranking pipeline without losing its quality gain
  • How to remove organizations that match a local-search query by a formal feature but not by meaning?
  • How to serve a heavy BERT relevance model in a latency-bound ranking pipeline?

Tools and methods

  • DSSM — fast two-tower student that predicts the teacher's verdict
  • CatBoost — ranking model (about 600 features) and post-filtering models
  • Knowledge distillation — moves the teacher's verdicts into the student
  • Expert relevance labels — the training target that worked
Continue reading — it's free

Google or a one-time email code. No password.