Building RAG Systems That Work Accurately: From Prototype to Thousands of Users a Day

Alexey Bychenkov23 May 2026

Problem

Employees of a large retailer lose hours searching for contracts, regulations and reports scattered across PDF, DOCX and XLSX files, scans of old paper and several storages. The slides cite McKinsey 2025: 9.5 hours a week per office worker, 23% of a 40-hour week. A naive RAG built in two weeks looked fine on 10 documents. At 50, 100 and 150 documents it returned a mess and made up answers.

Example from the article: one business unit had contracts from the 2000s that nobody could find. After the scans went through a VLM, they find them "with a click".

Solution in brief

X5 treats RAG as a pipeline where every stage must be clean, and fixed four weak spots in turn: fixed-size chunks (semantic chunks that never split tables), scans (a VLM), exact identifiers (hybrid dense plus BM25 search) and noisy top results (cross-encoder reranking). Streaming dialogue and night batch ingestion followed.

Questions this case answers
  • How to build a question-answering system over tens of thousands of heterogeneous documents with tables and scans, where exact identifiers and meaning must both be found and answers need citations
  • How to chunk documents and tables so retrieval does not split clauses or structure, and how to move a prototype that works on ten documents to production scale
  • How to build a RAG system over tens of thousands of heterogeneous corporate documents (PDF, DOCX, XLSX, scans) that answers with citations
  • How to chunk documents and tables so that retrieval does not lose clauses and structure

Tools and methods

  • Semantic chunking, table-aware — 450-token chunks with an overlap of 200, cut at paragraph or section ends
  • VLM, five processing modes — reads scans and drawings
  • Hybrid search with RRF fusion — dense 1024-dim vectors plus sparse BM25
  • BGE-reranker (cross-encoder) — re-scores the top candidates against the query
  • GraphRAG — entities and links instead of chunks
  • REST API and MCP plugin — expose the base to other products
Continue reading — it's free

Google or a one-time email code. No password.