Problem
A business asks an NLP team to launch RAG over corporate documents quickly. Internet advice is a pile of unrelated tricks, and it is unclear which steps matter. If the retriever passes wrong or too many documents, the LLM answers wrongly. The speaker's team at X5 Tech tackled this for internal regulatory documents in Russian.
Illustrative example: an employee asks about an internal regulation. The retriever returns 30 chunks, and the LLM must answer from them without drifting into its own knowledge or switching to Chinese.
Solution in brief
Build the pipeline stage by stage: chunk documents, retrieve with a good multilingual encoder (optionally plus BM25), re-rank candidates with a cross-encoder, then control generation through prompt structure and sampling parameters. Score retrieval on a benchmark set, treat LLM-based metrics as a side signal, and let experts validate answers.
Questions this case answers
- How to make an LLM answer strictly from retrieved documents when questions and documents live in different semantic spaces and no reference answers exist
- How to choose the retrieval stages of a document QA pipeline: chunking, hybrid first-pass retrieval and cross-encoder re-ranking of the shortlist
- How to build a RAG pipeline over corporate documents with reliable retrieval and answer quality
- How to improve retrieval quality in RAG with chunking, hybrid search and re-ranking
Tools and methods
- Qdrant — vector store in the team's final stack
- intfloat/multilingual-e5-large — bi-encoder used for retrieval (
clips/mfaqwas also tried) - Parent-document retrieval, BM25 ensemble, symmetric search by questions — retrieval variants
- Cross-encoder + boosting re-rank — re-ranking of the top-K candidates
- Llama 3 with role play, chain of thought, few-shot — generator and prompting
- Negative presence penalty, logits processor — sampling controls
- MAP@K, P@K, nDCG@K, groundedness, context relevance — evaluation
Google or a one-time email code. No password.
