YandexGPT as a RAG assistant for tech support: solving problems faster

Lena Wolf24 May 2025

Problem

Yandex support operators answer chats by searching long internal documents and reply templates. Some documents run to tens of thousands of tokens, a few to a million. Full automation is not an option: users bring cases missing from the documentation, angry or sensitive cases, and screenshots that lose quality in recognition. The team gave every operator a draft reply, so response time drops while the human stays in the loop.

Example from the article: a client writes "I want to cancel a paid subscription", and the knowledge base calls the product a "Membership". The assistant has to find the auto-renewal document without a word match.

Solution in brief

Retrieve chunks for the chat by embeddings, rerank them with a model trained on crowdsourced relevance labels, and let a fine-tuned LLM draft the reply from the top-3 chunks plus client metadata. Chunks follow the document hierarchy and keep templates whole. The model refuses when the documents lack an answer, and refusals are hidden from operators.

Questions this case answers
  • How to build a retrieval-augmented drafting assistant over long hierarchical documents that must refuse when the knowledge base has no answer
  • How to raise the hit rate of a retriever over hierarchical chunks with crowdsourced relevance labels and a reranker
  • How to build a RAG assistant that suggests replies to support operators from a knowledge base
  • How to chunk long hierarchical instructions and reply templates for retrieval

Tools and methods

  • Hierarchical chunking — keeps reply templates whole and grows each chunk up the heading levels within a token limit
  • Embedding retriever with a reranker — nearest chunks by embedding, then a learned reranking of the top-20
  • Crowdsourced labeling — chunk relevance and reference answers
  • Fine-tuned generative model — answers from the attached documents or refuses
  • hitrate@N — share of dialogues whose target chunk is in the top-N
Continue reading — it's free

Google or a one-time email code. No password.