Problem
Teams with a large video archive (a TV series, training videos, news clips, lectures) cannot find the moment where a name, fact or phrase was said. The only way is to scrub through clips by hand. Titles do not help: the question is about spoken words. A usable answer gives the video, the second where the phrase starts, and a visual check of the scene.
Example from the article: a search site for the French series "Bref" (82 episodes of about 2 minutes) answers a query such as "au revoir" with episode 57, second 65, a thumbnail of that frame and a two-second preview.
Solution in brief
Transcribe each video into subtitle lines with timestamps, and store every line as one record in a text search index. The record carries the video's metadata (ID, title, order, duration, views) and the line's start second. Frames and short previews cut at that second give the visual result. Views and a "distinct by video" setting control ranking and repeats.
Questions this case answers
- How to make a large archive of unstructured audio-visual content searchable by its spoken words with exact timestamps
- How to return one best-ranked hit per parent document when a long item contains many matching text fragments
- How to find the exact moment in hours of video where a phrase was spoken
- How to show a video search hit with a thumbnail and a preview of the exact scene
Tools and methods
- yt-dlp — downloads a playlist and dumps video metadata (views, heatmap) as JSON.
- FFmpeg — extracts audio and cuts the thumbnail and the two-second preview.
- HappyScribe — speech-to-text with .vtt subtitle output.
- Algolia — hosted search index with the Distinct setting.
- Cloudinary — image CDN that compresses media on the fly.
- Next.js on Vercel — the demo web front end.
Google or a one-time email code. No password.