Unified Context-Intent Embeddings for Scalable Text-to-SQL

Keqiang Li, Bin YangMar 2026

Problem

Pinterest has over 100,000 analytical tables and more than 2,500 analytical users. A question such as "What's the engagement rate for organic content by country?" rarely matches any table description. Several tables can answer it, but only specific join patterns work, and metrics follow company conventions. Trust signals and past query patterns sit in different systems, so one search cannot return all the context, and success depends on chance.

Example from the article: the question "What's the CPC for our top keywords?" has to match a query once written for "October 2024 keyword performance".

Solution in brief

Treat query history as a library of expert solutions. An LLM turns every historical SQL query into a text description of its analytical intent, which is embedded and searched by meaning. Join keys, filters and aggregations parsed from the same queries, plus usage statistics and table governance metadata, rank the results so that trusted tables come first.

Questions this case answers
  • How to ground a natural-language data assistant in the history of queries people already ran when table descriptions do not match the wording of questions
  • How to rank candidate data sources by trust signals as well as relevance when many near-duplicate sources exist
  • How to make a text-to-SQL agent pick the right tables when the analyst's question does not match any table description?
  • How to reuse the join keys, filters and metric logic that analysts already wrote instead of letting the LLM guess them?

Tools and methods

  • Unified context-intent embeddings — SQL plus catalog context turned into a text description of intent, then embedded
  • Structural and statistical patterns — joins, filters, co-occurrence and success rates from query logs
  • Governance-aware ranking — similarity fused with table tier and freshness
  • DataHub — open source catalog under PinCat (tiers, glossary terms)
  • OpenSearch — vector index with metadata filters, filled by Airflow jobs
  • Model Context Protocol (MCP) — tools for search and query execution
Continue reading — it's free

Google or a one-time email code. No password.