Production Patterns for Multi-Tenant AI: Scaling to 800 Banks on Databricks

Natasha Ueberschlag, Simon Schmitz18 Jun 2026

Problem

Atruvia, the IT provider of about 800 German banks, built the Datenraumbot, a RAG assistant over each bank's documents. Every bank is a separate legal entity, so no data can be shared between tenants. The first version (on-prem Postgres with pgvector) worked for about 50 banks, but every new database needed another team to provision it, so it did not scale to 800. Top-K retrieval also cannot answer whole-knowledge-base questions such as "what do you know?".

Example from the article: a newly onboarded user asks "what do you know?" and a plain retrieval assistant answers with random details (slide 15) instead of an overview.

Solution in brief

Each bank gets one vector index named from its bank ID, so one agent serves all tenants and isolation sits at the index level. Three Delta tables and a scheduled Spark pipeline place the indexes on endpoints. For holistic questions, the team builds a RAPTOR knowledge tree of document and cluster summaries offline and searches it with one flat similarity search.

Questions this case answers
  • How to serve question answering over documents to hundreds of isolated tenants without per-tenant infrastructure and with a fixed limit on index count
  • How to answer whole-corpus overview questions that top-K chunk retrieval cannot answer
  • How to serve RAG over documents to hundreds of isolated tenants without per-tenant infrastructure
  • How to spread hundreds of vector search indexes across endpoints with a fixed index limit

Tools and methods

  • Databricks Vector Search — one index per bank
  • Unity Catalog — schema pattern bank_*_schema as the tenant registry
  • Delta tables and Spark — status, capacity and failures tables; parallel provisioning
  • RAPTOR with Gaussian mixture clustering — recursive summary tree for holistic questions
  • text-embedding-3-large and cosine similarity — embeddings for the tree and for citation matching
  • RAGAS metrics — offline evaluation
Continue reading — it's free

Google or a one-time email code. No password.