Full talk · edited transcript

RLHF for retrieval based NLP system

Alexander Abramov27 May 2023

Alexander Abramov, Sber Devices (chief technology expert, ML Team Lead; the team works with NLP models, natural language understanding, and retrieval-based models)

Topic of the presentation: Reinforcement Learning from Human Feedback for retrieval systems. The approach is suitable not only for NLP, but also for recommendations and any search tasks where embeddings are present.

Presentation plan: base models and training methods, data (where they come from, what hacks and tools are available), environment for RL experiments with retrieval, use of critic models and their robustness against blackbox attacks, then classical RL for retrieval (two algorithms and transition from one to another, what retrieval approaches can be adopted and from where), bonus about orchestrating entire modules on different models using multi-armed bandits, and finally the impact of all this on metrics compared to what would be the case without RL.

Continue reading — it's free

Google or a one-time email code. No password.