Method
Side-by-side evaluation
Pairwise comparison of two outputs by humans or LLM judges.
17 cases from 10 companies apply Side-by-side evaluation. 2 were checked in an online A/B test; 7 report results in production.
Typical tasks: Evaluate models, Rank search results, Build an assistant over internal docs, Label data with LLMs and 2 more. Companies: Yandex, Sber, Databricks, LinkedIn, DoorDash, T-Bank and 4 more. Often combined with: LLM-as-a-judge, Relevance labeling, Knowledge distillation, RAG, Supervised fine-tuning and 46 more.




