Problem
An online cinema sells its catalog through poster and title tiles on the main page. The rights holder supplies one poster per title, so users see the same image again and again and stop reacting to it. A recommender could show different posters to different users, but nobody has the alternatives. Designing them by hand for thousands of titles does not scale.
Example from the article: IVI built the Parker pipeline, which turns a film or cartoon into a batch of posters with metadata for the recommendation system.
Solution in brief
Treat every frame with a face as a poster candidate. Detect faces, recognize actors by grouping faces first and classifying the groups, then cut the pool with cheap staged scoring: blur, aesthetics, "absurd" expressions. Crop, upscale and recolor the survivors, and let editors pick several final posters per title.
Questions this case answers
- How to generate several candidate images from video so a recommender can show different ones to different users, when only one asset is supplied
- How to rank and filter a huge pool of candidate frames by subjective quality when no ground-truth quality labels exist
- How to identify people in video when the full identity set is large but the candidate set per video is small and known
- How to produce many alternative posters for a film when the rights holder supplies only one?
Tools and methods
- YOLO — face detection for films and cartoons in one model
- ArcFace embeddings — 512-dimensional face vectors for actor recognition
- Toloka — crowd labeling of frame aesthetics and facial expressions
- OpenCV — Farneback optical flow for motion blur
- torchvision — pretrained classifiers for the "absurd face" model
- NVIDIA Triton and ONNX — serving models across several GPUs
Google or a one-time email code. No password.