Advanced RAG Pipelines

Ivan Nasonov25 May 2024

Problem

Teams that put an LLM over their own documents hit the limits of plain RAG. A large context window does not help: the talk cites late-2023 articles showing that models attend to the start and end of the context and poorly remember the middle. Basic vector search then drowns in noise on huge bases, returns too many fragments for a general question, brings related facts nobody asked for, and cannot assemble an answer about a whole book.

Example from the article: a question about the children of Captain Grant needs the plot of the whole Jules Verne novel. Small fragments cannot cover it, and feeding in the whole book defeats the purpose of RAG.

Solution in brief

The talk is a catalogue of retrieval patterns, each tied to one failure of basic RAG. Hierarchies and graphs fix scale and structure, HyDE, Multi-Query and RAG-Fusion fix poor queries, compressors trim the output, and RAPTOR builds a tree of summaries for thematic questions.

Questions this case answers
  • How to answer questions over a very large document collection with retrieval when flat vector search returns noisy chunks and a question needs a whole collection summarized
  • How to rewrite or expand a vague user query before retrieval so that it matches the stored text
  • How to make RAG answers more accurate when the document base is very large
  • How to retrieve better context for a vague or too general user question

Tools and methods

  • Document hierarchies — search by clusters or chunk size, not the flat base
  • Graph RAG, text-to-Cypher — entities and links from a graph database
  • HyDE — search by an LLM draft answer
  • Contextual compressors and filters — keep only the needed fragments
  • Multi-Query, RAG-Fusion — query variants; the second merges lists by Reciprocal Rank Fusion
  • Multimodal RAG — one embedding space for text and images
  • RAPTOR — tree of recursive cluster summaries
Continue reading — it's free

Google or a one-time email code. No password.