The Search of the Future

Vladislav Leshchuk23 May 2026

Problem

A team at Sber AI wanted an LLM that looks up documents with its own queries. A plain RAG pipeline cannot: it always runs the same retrieval, cannot rephrase a query and cannot tell whether the model already knows the answer. Prompt-based agents only tune the prompt and break on unusual queries. Reinforcement learning (RL) needs a lot of interaction data and a stable environment. The team also had to pick between cheap and costly retrieval.

Example from the article: base GigaChat 3.1-10b-a1.8b returned valid JSON in 10% of cases and skipped search in 35% of queries.

Solution in brief

The team compared BM25, embedding search and a DeepSeek search agent by nDCG@K on HotpotQA. Then they fine-tuned a small model with supervised fine-tuning (SFT) on search traces from Search-R1, an RL-trained model acting as teacher. A trace records the whole path to the answer, so the student sees when and how to search.

Questions this case answers
  • How to teach a small model to decide by itself when and what to query a search tool when reinforcement learning is too data-hungry and fixed retrieval cannot rephrase or skip search
  • How to choose between lexical, embedding and agent-based retrieval by ranking quality, latency and cost on one benchmark
  • How to teach a small LLM to call a search tool by itself with SFT on teacher traces
  • How to choose between BM25, semantic search and an LLM search agent by quality, latency and cost

Tools and methods

  • BM25, embedding search, DeepSeek agent — retrieval options compared on quality, time and cost
  • nDCG@K (K usually 2 or 5) — retrieval metric that accounts for the position of the right document
  • Search-R1 — RL-trained teacher that generates traces
  • GigaChat 3.1-10b-a1.8b — student model, fine-tuned with LoRA/QLoRA
  • FAISS index over a wiki corpus — retrieval backend
  • HotpotQA, NQ, TriviaQA, Bamboogle — evaluation datasets
Continue reading — it's free

Google or a one-time email code. No password.