Problem
Pinterest retrieves ads with online approximate nearest neighbor (ANN) search, which brought double-digit gains on ads quality metrics. The ads inventory keeps growing, while online latency and infrastructure cost must stay flat. Moving the index from HNSW to the inverted file (IVF) algorithm let one tier hold more than 10x more ads, but the cost increase was still too high to use that larger inventory.
Example from the article: Pinterest compared offline and online ANN for similar-item ads and for visual-embedding ad candidates, and online ANN was the costlier option.
Solution in brief
When the query side is static or limited to a known set of keys, run the ANN search in a batch job and store the neighbor lists in a key-value store. Serving then does a key lookup instead of a vector search, followed by an ID-based retrieval query. Pinterest reports up to 80% lower infrastructure cost.
Questions this case answers
- How to keep serving cost and latency flat when the candidate pool for embedding retrieval grows several times over
- How to replace per-request nearest-neighbor search with precomputed neighbor lists when the query side is a bounded set of keys
- How to cut the infrastructure cost of ANN candidate retrieval for ads when the ads inventory keeps growing?
- How to decide which candidate generators can use precomputed (offline) nearest neighbors instead of an online ANN index?
Tools and methods
- Offline ANN workflow — batch ANN search over a prepared list of query embeddings, with results stored as
<key, ANN neighbors>pairs. - Key-value store — holds the precomputed neighbor lists; the serving system reads it by key.
- ID-based retrieval query — fetches ad candidates from the indexing system using the stored neighbor IDs.
- IVF index — replaced HNSW in the online ANN index.
Google or a one-time email code. No password.