How We Do RL for Our LLM at Avito

Albert Akopyan23 May 2026

Problem

Avito runs more than 25 services on LLMs, including assistants for buyers and for sellers. LLM inference is expensive. One way to cut it is to make the model generate fewer tokens for the same text. The Core LLM team replaces the tokenizer of open models (Qwen3 8B, Qwen3 32B, Qwen3-Next 80B-A3B) so that Russian and Avito texts take fewer tokens. After the swap and pretraining, the model must match or beat the original Qwen, and post-training is the stage that delivers that.

Example from the article: Avito post-trains the new-tokenizer model with GRPO on three tasks with a 0/1 reward: math, instruction following and function calling.

Solution in brief

Swap the tokenizer, run pretraining, then SFT, then GRPO with rule-based verifiable rewards, one domain at a time. Each domain gets a dataset and a Python checker that returns 0 or 1. Online rollout filtering (DAPO) keeps batches informative, and a move from TRL to verl makes MoE training practical.

Questions this case answers
  • How to cut the inference cost of an LLM on a non-English language by changing its tokenizer and then restoring quality with reinforcement learning on automatically checkable rewards
  • How to generate synthetic multi-turn tool-calling dialogues for a set of internal tools
  • How to cut LLM inference cost for Russian text by swapping the tokenizer of an open-weights model
  • How to restore and improve instruction following, function calling and math after a tokenizer swap with RL on verifiable rewards

Tools and methods

  • GRPO / DAPO — RL algorithm; DAPO filters rollouts online
  • RLVR — rewards from Python checkers, binary 0/1
  • Qwen3 8B / 32B, Qwen3-Next 80B-A3B — base models with a replaced tokenizer
  • TRL with DeepSpeed, verl with Megatron — training frameworks for dense and MoE models
  • vLLM, SGLang — rollout generation
  • APIGen-MT-style pipeline, Persona Hub — synthetic multi-turn tool-calling dialogs
Continue reading — it's free

Google or a one-time email code. No password.