Full talk · edited transcript

Why Reports Mislead, or How to Measure the Effect of an LLM Feature

Nikita Bukhanchenko23 May 2026

Nikita Bukhanchenko (Sber, Tech Lead @ GigaChat Pretrain; previously head of delivery pricing at Avito)

The talk has two parts: how, in an ideal world, the effect of a new feature should be measured, and how it should not be measured. Part 1 ends with takeaways; part 2 goes through five mistakes.

Continue reading — it's free

Google or a one-time email code. No password.