Problem
Yandex Music shows a short trailer for a track, album or playlist, so a listener can judge it without playing the whole thing. A good trailer is recognizable, short, characteristic of the track and does not stop mid-word. Artists care which fragment represents their release, and they notice a wrong pick or a missing trailer quickly. The ML engineer has to turn each track into two numbers, trailer_start and trailer_end, for a catalog of very different music.
Example from the article: two trailers for the same song were played, and the better one was the one where the song title is sung right away.
Solution in brief
One neural network predicts beats, bars, structural segments and a custom "trailerness" score (whether a trailer can start here), each with its own output head. The trailer starts near the trailerness peak, snapped to beats and bars, and ends 8 bars later.
Questions this case answers
- How to pick a short representative excerpt of a long audio item and align its start and end to the item's temporal structure
- How to find out why a generated excerpt is wrong or missing in a multi-stage model pipeline
- How to automatically pick a short, recognizable preview fragment of a music track
- How to align a preview start and end to beats and bars so it does not cut off mid-phrase
Tools and methods
- Multi-head neural network — one model with a head per annotation type: beats, bars, segments, trailerness
- Open music datasets — beat, bar and segment annotation, with annotation and audio unified
- Own data — generative music plus annotation by live people, mostly for trailerness
- Time-stretch augmentation — random speed-up and slow-down of tracks
- Human evaluation — the main way to assess trailer quality
Google or a one-time email code. No password.