Method
Knowledge distillation
Training a smaller model to imitate a larger one or LLM labels.
53 cases from 27 companies apply Knowledge distillation. 17 were checked in an online A/B test; 21 report results in production.
Typical tasks: Rank search results, Reduce ML cost, Retrieve and rank from millions of items, Evaluate models and 11 more. Companies: Yandex, Pinterest, Meta, DoorDash, LinkedIn, Avito and 21 more. Often combined with: Supervised fine-tuning, LLM-as-a-judge, Multi-task learning, Two-tower model, LLM labeling and 125 more.



