Nikita Martynov (head of AGI NLP, Sber), Daniil Astafurov (principal engineer, AGI NLP development, Sber)
POLLUX is a project of Sber's AGI NLP team for measuring the generative capabilities of large language models in Russian. It includes two LLM-as-Judge models (7B and 32B), a dataset of expert ratings (in the speech "408 thousand", on slide 3 476K, on slide 28: 489,375 pointwise ratings), a typology of generative tasks, and a typology of criteria.
Continue reading — it's free
Google or a one-time email code. No password.