Problem
The World Bank Group built separate Genie spaces (Databricks natural-language-to-SQL assistants) domain by domain, and users had to know which space owned which question. Real questions cross domains, and answers also live in tens of millions of project documents and hundreds of Tableau, Power BI and Databricks dashboards. Genie on a plain gold layer gave different numbers for the same question. The bank judged even 80% correct unacceptable for structured data and did not launch.
Example from the article: "What projects did we execute in Kenya in 2022 where the risk rating is high, and why?" needs a project list from structured data, the reasons from project documents, and a chart.
Solution in brief
Ground each Genie space in a certified metric view so answers are deterministic and reconcile with dashboards. Put a LangGraph ReAct supervisor on top. Guided by a domain taxonomy, it decomposes questions, calls the right Genies, adds document-retrieval and visualization agents, and synthesizes one answer. MLflow judges and a weekly feedback loop keep it accurate.
Questions this case answers
- How to answer cross-domain questions by routing them to several scoped text-to-SQL spaces and combining their results with document retrieval so that the numbers match the existing dashboards
- How to evaluate and keep improving a multi-agent analytics assistant after launch with judges and a weekly feedback loop
- How to answer cross-domain analytics questions with an LLM agent that returns the same numbers as the dashboards
- How to route a natural-language question to the right domain text-to-SQL space and combine several answers into one
Tools and methods
- Genie spaces on metric views — one expert per domain or sub-domain, with deterministic answers
- LangGraph supervisor (ReAct) — think, act, observe, answer; routes using the domain taxonomy
- Databricks Vector Search (RAG) — retrieves the "why" from project documents
- Visualization and report-discovery agent — picks a chart type and surfaces existing BI reports
- MLflow — traces, prompt registry, offline LLM judges, online scorers
Google or a one-time email code. No password.