Tool

Toloka

22 cases from 14 companies use Toloka in their stack. 2 were checked in an online A/B test; 8 report results in production. Tasks: Label data with LLMs, Evaluate models, Rank search results, Match entities and 4 more. Methods: Crowdsourced labeling, Supervised fine-tuning, Overlap annotation, Clustering, Relevance labeling and 57 more. Companies: Yandex, Alfa-Bank, SberDevices, Sber, VK, Samokat.tech and 8 more.

Optimizing Relevance When Sorting by Price in Marketmarketplace2025-05
YandexART: Humans and Neural Networks in Evaluating Generative Modelssearch-engine2025-05
Labeling Product Cards with LLMse-commerce2025-05
The Entity Resolution Task: Atomic Factoryadtech2025-05
AI-Generated Music: Quality Evaluation Methodologies and Generation Optimizationmedia-streaming2025-05
LLM in Education: How We Created the Code Guardianedtech2024-05
Data Labeling: Experts, Crowdsourcing and Large Language Modelsgeneral2024-05
How We Trained a Content Video Encoder Without Labeled Datasocial2024-05
End2End: What to Think About If You Want to See Your Work in Production One Day (Problem and Solution Cases)edtech2024-05
Bayesian Truth Serum in Dataset Labeling: Finding the Truth When There Is Nonegeneral2024-05
Phone Number Scanner—banking2023-05
Clean Labels for MLsearch-engine2023-05
Automating New Product Onboarding in a Marketplace with MLmarketplace2023-05
Automating Topic Modeling with AutoTMgeneral2023-05
NLP at Alfa-Bank, Part 2: Data Labelingbanking2023-05
NLP at Alfa-Bank, Part 9: Structuring OFD Receiptsbanking2023-05
Beyond global sentiment: object, aspect sentiments—general2023-05
Majority vote for CV annotations: improve your data labeling pipelinegeneral2023-05
Finding key features of organizations in reviews on Yandex Mapssearch-engine2022-06
Building a Relevance Model for Text and Video Informationmedia-streaming2022-06
Poster generation for films and cartoons in an online cinemamedia-streaming2022-06
Building text summarization for Russian-language textsgeneral2022-06