Method

Self-consistency / best-of-N

Sampling several answers and choosing by voting or reranking.

7 cases from 7 companies apply Self-consistency / best-of-N. 3 report results in production.

Typical tasks: Evaluate models, Label data with LLMs, Rank search results, Retrieve and rank from millions of items and 1 more. Companies: HSE University, Netflix, Etsy, Yandex, Spotify, SberDevices and 1 more. Often combined with: LLM-as-a-judge, Supervised fine-tuning, Uncertainty quantification, Golden set / benchmark, Chain-of-thought and 22 more.