Evolving Recommendations: A Personalized User-Based Ranking Model

Yingji Pan, Bing Zhou, Yuri M. Brovman3 Oct 2023

Problem

eBay's homepage, feeds and personalized modules have no input item, only the user's shopping behavior. After candidate retrieval, a few hundred items must be put in a final order. The previous order came from a heuristic blending strategy that does not learn from feedback. A ranker trained on clicks alone or purchases alone is wrong about the other signal.

Example from the article: a user sees item A (not clicked), item B (clicked) and item C (purchased). The right order is C > B > A. A click-only model misorders B and C, and a purchase-only model cannot learn that B beats A.

Solution in brief

Train a wide and deep neural ranker with a pairwise loss on three label levels (purchased, clicked, neither). Feed it pretrained user and item embeddings from the retrieval model and derived behavior features such as freshness of interest. The wide part is a shallow network, not a linear model.

Questions this case answers
  • How to rank a few hundred retrieved candidates for a user when only user context is available and several engagement signals of different strength must be optimized together
  • How to combine hand-built behavioral features with pretrained user and item embeddings in one ranking model and attribute the offline gain to each part
  • How to rank hundreds of candidate items for a user when there is no input item and only the user's history is available?
  • How to optimize click and purchase probability in one ranker instead of choosing one of them?

Tools and methods

  • Wide and deep network — deep part generalizes over all features, shallow wide part keeps strong user-behavior features
  • Pairwise ranking loss — learns relative order of items shown to the same user, using purchase, click and no-action labels
  • Two-tower model — the existing personalized recall model, reused to produce user and item embeddings as ranker features
  • TorchScript — exported trained model served on a GPU inference platform
  • Apache Airflow — scheduler for training on eBay's GPU cluster
Continue reading — it's free

Google or a one-time email code. No password.