Building a Large-Scale Recommendation System: People You May Know

Parag Agrawal, Viral Gupta, Aman Gupta, Aastha Nigam6 Feb 2024

Problem

A recommender with a candidate inventory of billions of items cannot score every item with a heavy model in a reasonable time. Cheap sampling tricks (negative sampling, adaptive importance sampling, hierarchical softmax) speed up training, but scoring at inference stays the bottleneck. The final list must also balance several goals: the chance an invitation is sent, the chance it is accepted, fairness and diversity.

Example from the article: LinkedIn's "People You May Know" (PYMK) serves more than one billion members and processes hundreds of terabytes of data and hundreds of billions of potential connections daily.

Solution in brief

Split ranking into a funnel of stages, each shrinking the pool for the next one: candidate generation (L0), a light ranker (L1), a rich ranker (L2) and re-rankers. Each stage gets its own model class and its own offline metric, and the whole system is judged by A/B tests.

Questions this case answers
  • How to rank a candidate inventory too large to score with a heavy model within a latency budget while keeping the most relevant items
  • How to merge candidates from heterogeneous sources and several objectives into one final list that is also fair and diverse
  • How to score a candidate pool of billions of items with heavy models without blowing the latency budget?
  • How to combine candidates from graph, embedding and heuristic sources into one comparable ranking?

Tools and methods

  • Candidate generation (L0) — graph walks, embedding-based retrieval and heuristic sources cut billions of items to a few thousand
  • Light ranker (L1) — logistic regression or XGBoost calibrates candidates from different sources onto one scale
  • Rich ranker (L2) — deep neural networks predict the probability and value of each engagement event
  • Re-rankers — fairness, diversity and power-user re-rankers, with Bayesian optimization of objective weights
  • A/B tests — the final judge of the whole system
Continue reading — it's free

Google or a one-time email code. No password.