Method

Crowdsourced labeling

Crowd annotation with overlap, honeypots and label aggregation.

16 cases from 9 companies apply Crowdsourced labeling. 7 report results in production.

Typical tasks: Evaluate models, Label data with LLMs, Build an assistant over internal docs, Match entities. Companies: Yandex, SberDevices, Thumbtack, Wildberries, Alfa-Bank and 4 more. Often combined with: Supervised fine-tuning, LLM labeling, Overlap annotation, Clustering, Human in the loop and 38 more.