Task

How to evaluate models

Measure the quality of ML and LLM models or agents offline and in production: metrics, LLM judges, benchmarks, monitoring.

How Pinterest Improved Analytics Agent Compliance with Loop Engineeringsocial2026-11
Fact-Checking at Scale: Building a More Accurate and Efficient Synopsis Factuality Systemmedia-streaming2026-11
Agentic Credit Analysisfintech2026-10
Workbench: Building an AI Content Generation and Quality Assurance System at Scalemarketplace2026-10
Building Gavel: AI evaluation infrastructure across Eternalfood-delivery2026-10
The Model Is Not the Modelingfintech2026-09
Automated merchant review summary system with integrated feedbackfood-delivery2026-09
Inside Vera, DoorDash’s Data Agentfood-delivery2026-09
The Lifecycle of LLM-as-a-Judge: Building, Aligning, and Monitoring at scalemedia-streaming2026-09
How to improve Algolia search quality for content-heavy sitesgeneral2026-09
Eval-driven development: Lessons from evaluating GenAI at scaletravel2026-08
Building Jarvis Pro: Route first, answer laterfood-delivery2026-08
Automating Search Relevance Assessment at Scale with LLM-as-a-Judgee-commerce2026-08
Calibrating LLM-Based Population Estimates with Human Validationmarketplace2026-08
How We’re Building Scam Alert on WhatsApp With End-to-End Encryption and Verifiability Guaranteessocial2026-08
From weeks to a day: how we made LLM evaluation fast enough to iterate ontravel2026-07
Bootstrap Confidence Intervals for LLM Evaluationgeneral2026-07
Building Food Metadata with LLM Juries, Context Optimization & Multimodal AIfood-delivery2026-07
Building Ask DoorDash (Part 3): Evaluationfood-delivery2026-06
Agentic Risk Operationsfintech2026-06
A Human-Augmenting Agentic Workflow for Causal Inferencemedia-streaming2026-06
The 52x Multiplier: How Zepto Mastered AI Agent Evaluation at Scalee-commerce2026-06
Building Higher Quality Genie Agents (fka Genie spaces): How to Ensure Accuracy and Trustgeneral2026-06
Behind the Curtain: How We Do Eval in Geniegeneral2026-06
Stop Guessing, Start Scoring: LLM Extraction Evaluation on Databricksgeneral2026-06
Supercharging Genie Performance With Metric Viewssaas-b2b2026-06
Doubling Medical Safety: Fine-Tuning Open LLMs for Women's Health Without Human Labelshealthtech2026-06
How To Be an Agentic Analystgeneral2026-06
Sponsored by: Omni | AI analytics at Condé Nast: What it took to get to productionadtech2026-06
BI for AI: Transforming OTel Agent Telemetry into Enterprise Analyticsgeneral2026-06
Inside DoorDash’s one-click simulation and evaluation platform for support chatbotsfood-delivery2026-06
User-Guided LLM Answer Quality Evaluationsearch-engine2026-05
How we build ML pipelines for pathomorphology: data, annotation, agentshealthtech2026-05
AI Assistant for Salesclassifieds2026-05
An AI Tutor and Methods for Evaluating Itedtech2026-05
How We Are Deploying AI at a Universityedtech2026-05
How We Evaluated OCR on Russian-Language Documentsgeneral2026-05
HGRPO: Hierarchical Grouped Reward Policy Optimization for Multi-Turn Conversational Agentsgeneral2026-05
Developing a RAG System on the History of Bashkortostan Using a Benchmarking Methodologygeneral2026-05
You Need LLM-as-a-Judge! How to Stop Evaluating by Eye and Set Up Validationgeneral2026-05
AutoJudge: Dynamic Generation of LLM-as-Judge for Validating Agentic Pipelinesgeneral2026-05
Building a Battle-Tested Open-Source Agentic RAGgeneral2026-05
Method of Automated Construction of Production Digital Twins Using LLM Agentsmanufacturing2026-05
When Metrics Lie: Countering Domain Shift in MRIhealthtech2026-05
Solving the Task: Life Hacks from the Task Author (AvitoTech ML Cup 2026)classifieds2026-05
An AI Agent as Teacher and Examineredtech2026-05
Analyzing Public Pedagogical Benchmarks for AI Tutors and Building Synthetic Students for Testingedtech2026-05
How to Safely Roll Out New Versions of Product AI Agents Using an Auto-Metrics Systeme-commerce2026-05
When Models Should Stay Silentgeneral2026-05
Crosscheck: Benchmarking AI models in the real worldsocial2026-05