Method
Human in the loop
Human review or approval inside an automated pipeline.
62 cases from 35 companies apply Human in the loop. 8 were checked in an online A/B test; 28 report results in production.
Typical tasks: Evaluate models, Catch fraud and abuse, Moderate content at scale, Answer questions over data and 17 more. Companies: DoorDash, LinkedIn, Gusto, Ramp, Thumbtack, Netflix and 29 more. Often combined with: LLM-as-a-judge, Threshold selection, Rule-based heuristics, Embedding similarity matching, Golden set / benchmark and 105 more.






