How Large Language Models Help Users Formulate Queries in Yandex Maps

Anton Nekhaev24 May 2025

Problem

Yandex Maps wants to grow its discovery audience: users who browse for "a restaurant with a nice view" instead of one named place. Query suggest (autocomplete) is the top of that funnel, and it runs at over 10k RPS with q99 latency of tens of milliseconds. The team added text suggestions and tuned for recall, which raised successful discovery sessions, but junk appeared: 60% of suggestions were irrelevant and 42% per query were duplicates. There were no assessors for suggest.

Example from the article: for the query "italyanskaya" ("Italian") the suggest returned tiles and thermal baths instead of cuisines and restaurants, and "bosonozhki" ("sandals") returned temples and churches.

Solution in brief

Use an LLM as a measurement instrument: a relevance and duplicate labeler fine-tuned on ChatGPT labels. Then fix the pipeline with a two-tower retrieval and HNSW index, distill the labelers into cheap filters, and rewrite click-based query clusters with a generative LLM.

Questions this case answers
  • How to measure the quality of query autocomplete suggestions offline when there are no human assessors
  • How to retrieve query suggestions by meaning with a two-tower model under tight latency and throughput limits
  • How to remove irrelevant and duplicate suggestions from search autocomplete
  • How to measure autocomplete quality offline without human assessors

Tools and methods

  • LLM-as-judge instruments — binary relevance and duplicate classifiers, ChatGPT labels then fine-tuned YandexGPT
  • Two-tower LLM retrieval — query and suggestion embeddings from MASK-token representations, contrastive loss on clicks
  • HNSW index — approximate nearest-neighbor search over all suggestions
  • Distillation — LLM query tower into an MLP over bag of n-grams on CPU; LLM labelers into filter models
  • Query clustering and generative summarization — 4000 clusters, an LLM writes the 10 best suggestions per cluster
Continue reading — it's free

Google or a one-time email code. No password.