How we built enterprise search to be secure and private

Ian Hoffman, Claire Bowman7 Mar 2025

Problem

A team that adds search and LLM answers over connected apps (Google Drive, GitHub) has to keep strict security promises. Copying external documents into your own index creates a second store that can go stale. Search results and LLM context must also never include anything the requesting user cannot already see in the source system. Customers and admins want to grant and revoke access to each source themselves.

Example from the article: Slack added Google Drive and GitHub to its search and AI Answers, with the rule that no external data is stored in Slack's databases.

Solution in brief

Skip the index. At query time, call each partner's public search API with the user's own OAuth token, then pass the returned, already permissioned results to the search UI and to an LLM through retrieval-augmented generation (RAG). Request only read scopes, let admins and users opt in and revoke per source, and keep the LLM inside your own trust boundary.

Questions this case answers
  • How to retrieve permissioned content from third-party systems for search and LLM answers when that content may not be stored or indexed by us
  • How to add search and AI answers over external apps without copying their data into our own storage?
  • How to make sure search results and LLM context contain only documents the requesting user may already see?
  • How to keep results from external systems fresh without building and syncing an index?

Tools and methods

  • Federated real-time search — query external search APIs per request instead of indexing their data
  • OAuth — a per-user token limits every call to what that user may do in the external system
  • RAG — supplies the LLM only with permissioned content at runtime, no training on customer data
  • Escrow VPC for LLMs — the model provider never sees customer data
Continue reading — it's free

Google or a one-time email code. No password.