Full talk · edited transcript

RAG Evaluation. Evaluate yourself. How to teach LLM to evaluate LLM work

Vladislav Shufinsky25 May 2024

Vladislav Shufinsky, data analyst, Magnit Tech.

The speaker is developing a chatbot for internal users of Magnit Tech. He came into Data Science through a passion for sports statistics, runs two Telegram channels about the application of DS in sports, and contributes to sports open-source libraries. The presentation plan: RAG evaluation problem, evaluation methods, RAGAS framework, ARES framework (less automated and more complex, but with higher quality than RAGAS), and a practical case: how to use LLM to evaluate whether a chatbot needs a spell checker.

Continue reading — it's free

Google or a one-time email code. No password.