Beyond Two Towers: Launching the 3-Tower Engagement Co-Train Model (Part 2)

Longyu Zhao, Gwendolyn Zhao, Peng Yan, Yuanlu Bai, Yuan Wang, Yao Cheng, Ang Xu, Zhaohong Han17 Sep 2026

Problem

Pinterest scores every ad candidate of a request in a lightweight ranking stage. After targeting, a typical request has hundreds of thousands of candidates (P99 above 200K on some surfaces). A two-tower model is cheap: ad embeddings are cached offline and scoring is a dot product. But it cannot capture how a user's recent behavior interacts with a given ad. A heavier model applied to every candidate breaks the latency budget: scoring all candidates with the three-tower model pushed P99 inference past the 70 ms timeout.

Example from the article: a three-tower model with cross-attention cut offline loss by roughly 30%, but at first its P99 inference took over 200 ms.

Solution in brief

Train one model with two heads: a fast two-tower score for all candidates and a richer three-tower score (cross tower plus inter module) for the top 40K candidates by utility. Cache request-independent work and trim cross-attention cost with four serving changes.

Questions this case answers
  • How to add a heavier interaction model to a cheap dot-product ranker when the candidate set is too large to score every item within the latency budget
  • How to serve a cheap and an expensive prediction from one model so the expensive one runs only on a selected subset of candidates
  • How to add a more expressive ranking model on top of a two-tower model when each request has hundreds of thousands of candidates?
  • How to keep P99 model latency under a hard timeout after adding a cross-attention model to candidate scoring?

Tools and methods

  • Two-tower co-train head — fast scores for every candidate.
  • Cross tower — cross-attention between user sequences and candidate features.
  • Inter module — shared-bottom MLP with logits for three tasks.
  • BF16, late user-feature expansion, pre-layer normalization — cut latency.
  • Two-stage scoring with utility topK — the costly score runs on a subset.
Continue reading — it's free

Google or a one-time email code. No password.