Method
LLM-as-a-judge
Using an LLM to grade outputs, with calibration and bias checks.
114 cases from 61 companies apply LLM-as-a-judge. 14 were checked in an online A/B test; 40 report results in production.
Typical tasks: Evaluate models, Build an assistant over internal docs, Rank search results, Answer questions over data and 18 more. Companies: DoorDash, Yandex, Instacart, Databricks, LinkedIn, Netflix and 55 more. Often combined with: RAG, Golden set / benchmark, Few-shot prompting, Human in the loop, Synthetic data generation (LLM) and 152 more.









