Launching Automated Video Moderation

Vladimir Morozov25 May 2024

Problem

Avito added video to listings and had to moderate it. Manual video moderation is expensive, slow, hard to scale with the flow and error-prone. Automating it started from poor data: video was new, violations are a small share, the labels were noisy and had no violation reasons, and videos with violations were sometimes deleted from storage.

Example from the article: an apartment listing with a video about May 1 and labor, caught as "video not about the item".

Solution in brief

Split a violation into a domain (frames, audio, metadata, link to the listing) plus a reason, and cover each domain with a simple, reusable detector: CLIP frame embeddings with a binary head per reason, OCR and ASR with regexes, logo and face matching. Measure online by sending a sample of "no violation" videos to manual review and scaling the result up.

Questions this case answers
  • How to launch automatic moderation of uploaded content with many violation reasons when labeled violations are scarce and noisy
  • How to estimate precision, recall and error rate of an automatic filter when manual review only sees a sample of the items it let through
  • How to launch automatic moderation of user-uploaded video when labeled violations are scarce and noisy
  • How to cover a new violation reason quickly without training data

Tools and methods

  • CLIP — frame embeddings, zero-shot detection and text-to-video matching
  • Joint Net — merges frame embeddings into one video embedding (Avg, Sum, Attention, CNN, RNN)
  • OCR and Whisper ASR with regular expressions — violations in on-screen text and speech
  • Detectors and matchers for logos and faces — compare embeddings with a database
  • Gradient boosting on metadata — filters out very low-quality video by bitrate and FPS
  • Sampler for manual review — estimates false negatives and true negatives
Continue reading — it's free

Google or a one-time email code. No password.