Method
Human / expert labeling
Manual annotation with guidelines and expert review.
58 cases from 39 companies apply Human / expert labeling. 8 were checked in an online A/B test; 26 report results in production. Includes variants: Overlap annotation.
Typical tasks: Evaluate models, Label data with LLMs, Moderate content at scale, Catch fraud and abuse and 11 more. Companies: Alfa-Bank, Celsus, Yandex, Airbnb, Avito, Sber and 33 more. Often combined with: Overlap annotation, LLM-as-a-judge, Golden set / benchmark, Supervised fine-tuning, Human in the loop and 110 more.












