Internships on the LLM Team: Several LLM Deployments at Avito

Olesya Moiseenko, Andrey Tokarev, Nikolay Khokhlov24 May 2025

Problem

Avito sells cars through products where a manager calls the client and then hands them to a dealer. A good call follows a checklist (introduce yourself, greet, agree on a meeting), and better calls raise the chance of a deal. Manual labeling covers only 5% of calls and is slow and expensive. The contractor's model was cheaper, but its quality was not always good. Checklists differ between products and dealers and keep changing, so a model must handle criteria it has not seen.

Example from the article: a transcript plus the criterion "the manager introduced themselves and greeted the client" goes in, and "yes" or "no" comes out.

Solution in brief

Fine-tune an LLM that takes a call transcript and one checklist criterion and answers yes or no, a single token each. The team compared per-checklist LoRA adapters with one general model trained on all checklists, and served the model with vLLM. The same recipe (large-LLM labeling with a validated prompt, LoRA training, vLLM service) was reused for support-reply rewriting and Autoteka abbreviation decoding.

Questions this case answers
  • How to automate yes-or-no checks of long conversations against a set of changing criteria with a small fine-tuned LLM
  • How to train a small model with a LoRA adapter on labels produced by a large LLM with a long validated prompt when no labels exist
  • How to automate call-quality checklist evaluation with a fine-tuned LLM
  • How to train a small LLM with LoRA on labels generated by a large LLM with a long prompt

Tools and methods

  • LoRA adapters — fine-tuning on a base LLM, per task or per checklist.
  • vLLM — model serving.
  • LLM-based labeling — a large LLM with a long prompt generates labels.
  • Redis cache — keyed with a model-version check.
  • Regex post-processing — output rules enforced after generation.
Continue reading — it's free

Google or a one-time email code. No password.