Establishing a Large Scale Learned Retrieval System at Pinterest

Bowen Deng, Zhibo Fan, Dafang He, Ying Huang, Raymond Hsu, James Li, Dylan Wang, Jay Adams31 Jan 2025

Problem

Pinterest Homefeed picks from billions of Pins with more than 20 candidate generators. Many retrieval sources were heuristics (Pin-Board graphs, user-followed interests), while ranking was already learned from raw engagement sequences. A learned retriever has two halves, a user model and a Pin model, deployed to separate services at different speeds. If the two versions differ, the embedding spaces mismatch and candidate quality drops sharply. A rollback on either side causes the same problem.

Example from the article: Pin indexing takes much longer than preparing the new viewer model, so during an index rollout some ANN hosts serve version N and others N+1.

Solution in brief

Train a two-tower model on logged engagement with sampled softmax and a popularity correction. Serve Pin embeddings from an ANN index and compute the user embedding per request. Each ANN host carries the model version of its index, and the backend uses the user model of that version.

Questions this case answers
  • How to retrain and deploy a two-tower retrieval model whose two halves ship through separate services without serving mismatched embedding spaces
  • How to train a large-scale embedding retrieval model on logged engagement with sampled negatives and correct the popularity bias this introduces
  • How to replace heuristic candidate generators with one retrieval model learned from engagement logs?
  • How to keep user and item embeddings from mismatching when the two towers deploy at different speeds?

Tools and methods

  • Two-tower model — user/context tower and Pin tower, scored by dot product
  • Sampled softmax with in-batch negatives — trains without a softmax over the whole corpus
  • Logit correction — removes popularity bias from sampling
  • HNSW-based ANN index (in-house Manas) — nearest neighbor search over Pin embeddings
  • Auto retraining with a validator — refreshes both towers and gates deployment
  • Model version metadata on ANN hosts — keeps user and Pin models in the same version
Continue reading — it's free

Google or a one-time email code. No password.