Problem
Teams that run a RAG assistant over documents (law, medicine, industry) still get answers with invented or distorted facts. Users trust fluent generated text, and the answer cannot be checked without rereading the sources, so the risk lands on production use in critical domains. Retrieval adds to it: cosine similarity over equal-size chunks misses relevant passages. The team needed a cheap score of how much of an answer the documents support.
Example from the article: an answer is split into sentences, and for each sentence the check returns the index of the supporting document sentence or -1, for example [-1, 7, 7, -1, -1, -1, 0].
Solution in brief
Split the answer and the documents into sentences or snippets, compute a similarity matrix, and keep for each answer sentence the best document sentence above a threshold. The share of answer text with no support is the hallucination estimate, and the same index shows what supports each sentence. Use the score to rank candidate answers or to refuse when too much text is unsupported.
Questions this case answers
- How to measure how much of a generated answer is supported by source documents by matching answer units to document units
- How to decide automatically to refuse a generated answer when too much of its text has no support in the sources
- How to detect hallucinations in RAG answers by checking which answer parts are supported by source snippets
- How to decide automatically that a RAG system should refuse to answer
Tools and methods
- Similarity matrix with threshold — maps each answer sentence to its best-matching document sentence
- SBert embeddings + cosine similarity — the Python variant in ExplainitAll
- n-gram Jaccard distance — the C# variant in AiFramework
- Cross-encoder (vectara/hallucination_evaluation_model) — third-party alternative the authors compare against
- Fractal answer synthesis — the authors' graph-based replacement for chunking, which feeds the same RAG
Google or a one-time email code. No password.