Problem
When a third-party media partnership expires, Canva must replace that partner's images in every library template. Replacing images by hand is a slow, manual process.
The replacement must look like the original, be safe to use under the company's intellectual property (IP) rules, and come from a library of more than 150 million images. It must also match the frame's aspect ratio and track library changes.
Example from the article: a template contains a photo of an apple on a table in a park. The designer needs another photo of an apple, ideally red, with a similar background and mood.
Solution in brief
Canva built an image-to-image search. Every library image becomes a DINOv2 embedding, picked after a manual comparison of five models. The embeddings sit in an external vector database with metadata filters, and the top 8 neighbors are shown to designers inside the template editor, who pick a suggestion before the design goes to human review.
Questions this case answers
- How to retrieve the most visually similar items from a very large catalog when text metadata misses visual nuance and no labeled similarity data exists
- How to choose between candidate embedding models for similarity search when only human judgment of nearest neighbors is available
- How to find visually similar replacement images for a large media library without manual searching?
- How to choose an image embedding model when there is no labeled similarity data?
Tools and methods
- DINOv2 — image embedding model chosen for production
- CLIP, ViTMAE, DreamSim, CaiT — other candidate embedding models, not chosen
- GPT-4o description + CLIP text search — tested alternative, the weakest
- Faiss — in-memory vector index used for the model comparison
- External third-party vector database — production store with metadata filtering
- Human-in-the-loop review — a person checks every replacement before republishing
Google or a one-time email code. No password.