Problem
Support specialists at a customer of Innotech spent time on typical requests that the corporate knowledge base already answered, because its keyword search worked poorly. The team had to build a search that finds the right documents and writes an answer from them. It had to index a new article within 15 minutes, answer within 10 seconds and fit on one GPU. The base held more than 5,000 articles plus Office and PDF attachments.
Example from the article: a user asks for the structure requirements of the "Administrator Guide" artifact and gets a point-by-point answer.
Solution in brief
A RAG service with a domain-tuned embedder, hybrid search (vector plus BM25 over title and content, merged by RRF), small chunks expanded with their neighbors, a cross-encoder reranker and an LLM for the answer. Offline, on questions written by the customer's analysts, MRR@5 rose from 0.22 (zero-shot baseline) to 0.802.
Questions this case answers
- How to answer questions over a large internal document base with a search that merges keyword and vector rankings whose scores are not comparable and indexes new documents quickly
- How to adapt a text embedding model to a narrow vocabulary without labeled query-document pairs
- How to replace a weak keyword search over a corporate knowledge base with RAG search that also generates an answer
- How to adapt an embedding model to a narrow domain without labeled queries
Tools and methods
- TSDAE, then GPL — two-stage unsupervised tuning of the embedder to the customer's domain.
- Hybrid search with RRF — vector search and BM25 over title and content, merged by rank.
- Small2Big chunking — search over small chunks, pass the chunk with its neighbors to the LLM.
- Cross-encoder or MMR — switchable reranking.
- MRR@5, chrF++, LLM judges — retrieval and answer metrics.
- OpenSearch, LangChain, Docling — storage and parsing.
Google or a one-time email code. No password.