Problem
An ads candidate generator built on a user's offsite conversion history computes the user embedding offline. At serving time it does not know what the user is viewing right now. On contextual surfaces such as Related Pins and Search, the current Pin or query is the strongest intent signal. Without it, the candidates lose in ranking and the auction: under 1% of Pinterest's Related Pins impressions came from this generator.
Example from the article: a user views a Pin of a "vintage leather armchair". The ads should match that item, not only the user's long-term interests.
Solution in brief
Add a context layer to the user tower and concatenate its output with the transformer's sequence output before the final MLP. Context is not logged with offsite conversions, so train with pseudo-context taken from the positive label. At serving, run the transformer offline, store its last hidden state, and run only the context layer and the MLP per request.
Questions this case answers
- How to make a retrieval model that encodes long user history respond to request-time context that is unavailable in the training data
- How to serve a heavy sequence encoder together with a light request-time context layer under a latency budget
- How to add real-time context to a sequential two-tower candidate generator when context is only known at serving time
- How to train a retrieval model on context features that are not logged next to the training events
Tools and methods
- Two-tower model — retrieval model with separate user and item encoders, trained with softmax loss
- Sequential recommendation — transformer encoder over the user's offsite conversion events
- Context layer — small network over request-time features, here the subject Pin's top interest categories
- Synthetic context augmentation — pseudo-context derived from the positive item during training
- Feature store — holds the daily-refreshed last hidden state of the user encoder
Google or a one-time email code. No password.