Problem
Atruvia, the IT provider of about 800 German banks, built the Datenraumbot, a RAG assistant over each bank's documents. Every bank is a separate legal entity, so no data can be shared between tenants. The first version (on-prem Postgres with pgvector) worked for about 50 banks, but every new database needed another team to provision it, so it did not scale to 800. Top-K retrieval also cannot answer whole-knowledge-base questions such as "what do you know?".
Example from the article: a newly onboarded user asks "what do you know?" and a plain retrieval assistant answers with random details (slide 15) instead of an overview.
Solution in brief
Each bank gets one vector index named from its bank ID, so one agent serves all tenants and isolation sits at the index level. Three Delta tables and a scheduled Spark pipeline place the indexes on endpoints. For holistic questions, the team builds a RAPTOR knowledge tree of document and cluster summaries offline and searches it with one flat similarity search.
Questions this case answers
- How to serve question answering over documents to hundreds of isolated tenants without per-tenant infrastructure and with a fixed limit on index count
- How to answer whole-corpus overview questions that top-K chunk retrieval cannot answer
- How to serve RAG over documents to hundreds of isolated tenants without per-tenant infrastructure
- How to spread hundreds of vector search indexes across endpoints with a fixed index limit
Tools and methods
- Databricks Vector Search — one index per bank
- Unity Catalog — schema pattern
bank_*_schemaas the tenant registry - Delta tables and Spark — status, capacity and failures tables; parallel provisioning
- RAPTOR with Gaussian mixture clustering — recursive summary tree for holistic questions
- text-embedding-3-large and cosine similarity — embeddings for the tree and for citation matching
- RAGAS metrics — offline evaluation
Google or a one-time email code. No password.