Problem
A team at Sber AI wanted an LLM that looks up documents with its own queries. A plain RAG pipeline cannot: it always runs the same retrieval, cannot rephrase a query and cannot tell whether the model already knows the answer. Prompt-based agents only tune the prompt and break on unusual queries. Reinforcement learning (RL) needs a lot of interaction data and a stable environment. The team also had to pick between cheap and costly retrieval.
Example from the article: base GigaChat 3.1-10b-a1.8b returned valid JSON in 10% of cases and skipped search in 35% of queries.
Solution in brief
The team compared BM25, embedding search and a DeepSeek search agent by nDCG@K on HotpotQA. Then they fine-tuned a small model with supervised fine-tuning (SFT) on search traces from Search-R1, an RL-trained model acting as teacher. A trace records the whole path to the answer, so the student sees when and how to search.
Questions this case answers
- How to teach a small model to decide by itself when and what to query a search tool when reinforcement learning is too data-hungry and fixed retrieval cannot rephrase or skip search
- How to choose between lexical, embedding and agent-based retrieval by ranking quality, latency and cost on one benchmark
- How to teach a small LLM to call a search tool by itself with SFT on teacher traces
- How to choose between BM25, semantic search and an LLM search agent by quality, latency and cost
Tools and methods
- BM25, embedding search, DeepSeek agent — retrieval options compared on quality, time and cost
- nDCG@K (K usually 2 or 5) — retrieval metric that accounts for the position of the right document
- Search-R1 — RL-trained teacher that generates traces
- GigaChat 3.1-10b-a1.8b — student model, fine-tuned with LoRA/QLoRA
- FAISS index over a wiki corpus — retrieval backend
- HotpotQA, NQ, TriviaQA, Bamboogle — evaluation datasets
Google or a one-time email code. No password.