LinkSage: GNN-based Pinterest Off-site Content Understanding

Jianjin Dong, Michal Giemza, Qinglong Zeng, Andrey Gusev, Yangyi Lu, Han Sun, William Zhao, Jay Ma20 Mar 2024

Problem

Most Pins on Pinterest link to an external landing page, yet ranking models know little about what those pages contain. Safety checks, ads, shopping and feed ranking all depend on it. The corpus is tens of billions of pages, so a solution must also be cheap to run and refresh. Teams differ in how large an embedding they can afford.

Example from the article: the page embedding is added as a feature to the Ads conversion ranking model, which gained 2.0-5.0% in conversion rate per impression.

Solution in brief

LinkSage is a transformer that embeds a landing page from its own text and image features plus those of its graph neighbours. The graph is built from (Pin, landing page) pairs that pass a semantic-agreement filter. One model is trained with a nested multi-size loss, and only newly crawled pages are re-embedded each day.

Questions this case answers
  • How to learn a reusable semantic embedding for a huge, constantly growing set of external documents when only noisy links between them and internal items are available
  • How to serve multiple embedding sizes from a single trained model so consumers can trade quality for cost
  • How to get one reusable embedding for every external landing page that ranking models can use as a feature?
  • How to build a content embedding for pages with little engagement data from a graph of curated links?

Tools and methods

  • Heterogeneous graph of (Pin, landing page) pairs — connects similar pages through the Pins that point to them.
  • Random walk with visit counts — picks and ranks each page's neighbours.
  • Transformer encoder — attends over page, neighbour and graph-structure features.
  • Contrastive training on Pin-page pairs — positives by a cohesion score, batch and random negatives.
  • Matryoshka Representation Learning — one model, five embedding sizes.
  • Incremental serving — embeds only the pages crawled each day.
Continue reading — it's free

Google or a one-time email code. No password.