Method

Evaluation regression gate

Automated eval runs gating releases in CI or nightly.

11 cases from 9 companies apply Evaluation regression gate. 7 report results in production.

Typical tasks: Evaluate models, Answer questions over data, Reduce ML cost, Explain a metric change and 5 more. Companies: DoorDash, Pinterest, Airbnb, Databricks, Gusto, Positive Technologies and 3 more. Often combined with: LLM-as-a-judge, Golden set / benchmark, Agent evaluation, Prompt optimization, Stratified sampling / assignment and 32 more.