Deploying LLMs in Moderation: From PoC to the Target Solution

Ilya Chumak23 May 2026

Problem

A marketplace that checks more than 20 million listings a day automates over 95% of checks, but the rest still needs several hundred moderators and hundreds of thousands of manual checks per day. Moderators need about six months to reach a quality plateau and make mistakes meanwhile, because the regulations are large and hard to memorize. Manual moderation scales badly.

Example from the article: Avito moderation regulations run up to 200 pages, and the team had to decide how to hand them to an LLM so that check quality stays at the manual level.

Solution in brief

Convert each regulation into a compact prompt, then optimize it on labeled data with GEPA instead of by hand. Run a self-hosted 32B model by default, fine-tune an 8B vision model for weak violations, and call the LLM only for listings the existing ML models already suspect. Guardrails run before the LLM, and low-confidence answers go back to human moderators.

Questions this case answers
  • How to turn a very long set of rules into a compact prompt that an LLM applies reliably, optimized on labeled data instead of by hand
  • How to combine existing classifiers with an LLM so that the expensive model runs only on suspected cases
  • How to replace manual content moderation checks with an LLM line that matches human quality
  • How to turn a 200-page moderation regulation into a prompt an LLM can apply

Tools and methods

  • GEPA (Genetic-Pareto) — automatic prompt optimizer using LLM reflection.
  • 32B LLM, Avision-8B — default model and fine-tuned vision model.
  • Unsloth, LoRA — 16-bit fine-tuning of the 8B model.
  • vLLM — inference with prefix caching and logits access.
  • Existing ML models — recall layer flagging suspected violations.
  • Guardrails — regular expressions, classical and embedding models before the LLM.
Continue reading — it's free

Google or a one-time email code. No password.