Author: Karina Romanova, NLP engineer (previously SberDevices, development of generative chatbots)
The speaker began studying the application of LLaMA, its pitfalls, and the tasks the model is worth applying to; the topic is close to her previous work with generative networks. She runs a Telegram channel where she analyzes new articles and shares experience, and often publishes her work in open source.
The talk is an overview of how LLaMA is structured and how it was fine-tuned on instructions and dialogues: Alpaca, Vicuna, Koala, WizardLM, StackLLaMA. Throughout the talk, the speaker discusses methods for collecting instructional datasets (self-instruct, Evol-Instruct) and techniques that speed up fine-tuning (LoRA, 8-bit quantization, packing).
Google or a one-time email code. No password.