Method

Golden set / benchmark

Curated reference questions or labels for evaluation and regression.

42 cases from 29 companies apply Golden set / benchmark. 2 were checked in an online A/B test; 22 report results in production.

Typical tasks: Evaluate models, Answer questions over data, Rank search results, Label data with LLMs and 12 more. Companies: DoorDash, Ramp, Databricks, Netflix, Grab, Yandex and 23 more. Often combined with: LLM-as-a-judge, Human in the loop, Semantic layer, Agent evaluation, Text-to-SQL and 81 more.

Agentic Credit Analysisfintech2026-10
Inside Vera, DoorDash’s Data Agentfood-delivery2026-09
The Lifecycle of LLM-as-a-Judge: Building, Aligning, and Monitoring at scalemedia-streaming2026-09
How to improve Algolia search quality for content-heavy sitesgeneral2026-09
Eval-driven development: Lessons from evaluating GenAI at scaletravel2026-08
Building Jarvis Pro: Route first, answer laterfood-delivery2026-08
How AI is transforming analytics at Grabfood-delivery2026-08
Agentic Risk Operationsfintech2026-06
The 52x Multiplier: How Zepto Mastered AI Agent Evaluation at Scalee-commerce2026-06
Building Higher Quality Genie Agents (fka Genie spaces): How to Ensure Accuracy and Trustgeneral2026-06
Behind the Curtain: How We Do Eval in Geniegeneral2026-06
From 700 Trillion Signals to Action: Building Production AI Agents for Real-Time Media Planningadtech2026-06
Atlassian's Journey of Using AI/BI Genie From Pilot to Production and What's Next?saas-b2b2026-06
From Power BI DAX to Metric View KPIs: How Mercedes-Benz Korea Built “Talk to Your Data” at Scalegeneral2026-06
How To Be an Agentic Analystgeneral2026-06
Your BI Copilot Is Blind - Crafting an Agent That Goes Beyond the Dashboardgeneral2026-06
From Prompt to Classifier: A Production Case Studysaas-b2b2026-06
About Anti-Fraud and Cats: How to Fight Fraud, Not Just Detect It Bettergeneral2026-05
From Chaos to Order: Extracting Structure from Unstructured Text with LLMsmarketplace2026-05
AI Agents for Optimizing Yandex Lavka's Businessfood-delivery2026-05
How we build ML pipelines for pathomorphology: data, annotation, agentshealthtech2026-05
How We Evaluated OCR on Russian-Language Documentsgeneral2026-05
Building a Battle-Tested Open-Source Agentic RAGgeneral2026-05
LLM-as-a-Judge: Evaluating natural language searchfood-delivery2026-05
Evaluating Netflix Show Synopses with LLM-as-a-Judgemedia-streaming2026-04
Financial Benchmarksfintech2026-03
How Etsy Uses LLMs to Improve Search Relevancemarketplace2026-01
How Pinterest Built a Real‑Time Radar for Violative Content using AIsocial2025-12
Creating AI agent solutions for warehouse data access and securitysocial2025-08
How To Build Agents Users Can Trustfintech2025-07
Analytics, Data, Metrics and Motivation: How to Build a Data Labeling Process in a Large Bankbanking2025-05
Catch Me If You Can: LLM Quality Control for Customer Servicegeneral2025-05
Embedders 2025: A New Vector in Text Representationretail2025-05
How We Build Avito's Main Product Metricclassifieds2025-05
How DoorDash leverages LLMs to evaluate search result pagesfood-delivery2025-04
Automated GenAI-driven search quality evaluationsocial2024-12
How to improve search without looking at queries or resultssaas-b2b2024-11
Unleashing the power of large language models at DoorDash for a seamless shopping adventurefood-delivery2024-10
Data Strategy in a Large Group of Companiestelecom2024-05
Data Labeling: Experts, Crowdsourcing and Large Language Modelsgeneral2024-05
Clean Labels for MLsearch-engine2023-05
Evolving DoorDash's Substitution Recommendations Algorithmfood-delivery2022-09