Problem
Sber branch employees show clients a presentation on a tablet and need answers about more than a hundred banking products, whose terms change often (for example, after a key rate change). Asking a colleague or calling the support line takes time during a client visit. The tablet keyboard pushes employees to write very short questions such as "rate" or "%". Traffic also holds typos, random letters and off-topic questions, and a plain RAG pipeline wastes slow LLM calls on them.
Example from the article: a one-word question "rate" asked while a credit card presentation is open has to be understood as a credit card question, not rejected as noise.
Solution in brief
Sber puts cheap models in front of RAG: a censor classifier, a two-level cache and a topic detector that picks which part of the knowledge base to search. Complex support questions go to a separate graph RAG agent that walks decision-graph memos.
Questions this case answers
- How to put cheap filters and caches in front of a retrieval-augmented language model so that only valid questions reach it
- How to route very short, ambiguous questions to the right subset of a growing knowledge base before retrieval
- How to build a fast RAG assistant for bank branch staff who type very short questions on a tablet
- How to filter noisy, off-topic and provocative questions before they reach a RAG pipeline without rejecting valid ones
Tools and methods
- Censor model (mDeBERTa-v3-base) — filters noise, runs on CPU
- Full-text and vector cache — exact match plus query-to-query search on E5-multilingual Large Instruct
- Cross-encoder (bge-reranker-v2-m3) and CatBoost — product topic of a question
- RAG Fusion — question rewritten into several product-specific variants
- Graph RAG agents — LLM walks memo graphs, asks clarifying questions
- Query autocomplete — nudges toward longer questions
- vLLM, FP16, continuous batching — serving speedups
Google or a one-time email code. No password.