How We Built a Multi-Task Canonical Ranker for Recommendations at Etsy

Yuqing Pan, Raphael Louca18 Apr 2023

Problem

Etsy has more than 100 million listings and hundreds of recommendation modules on web and mobile. Each module had its own ranker, trained only on that module's data. Maintaining hundreds of models and daily pipelines became too costly, and the cost slowed down adding new features, which could eventually degrade recommendation quality. The team also wanted to optimize for return visits, but a single session gives no direct signal that a buyer will come back.

Example from the article: two modules on the same item page, "more from this shop" and "you may also like", were each served by a separate ranker.

Solution in brief

Train one neural "canonical" ranker on data from a few representative modules and use it for many modules. It predicts two outcomes, favorite and purchase, and combines the scores. Favorite rate stands in for revisit frequency, and purchase is a guardrail. A module-name feature and sampling of training rows let one model work across modules.

Questions this case answers
  • How to serve many ranking surfaces with one shared model instead of one model per surface, without losing quality on any of them
  • How to optimize a ranker for a hard-to-observe long-term goal through a proxy while protecting the main business metric
  • How to replace hundreds of per-module recommendation rankers with one model that serves many modules?
  • How to optimize recommendations for return visits without hurting purchases?

Tools and methods

  • Multi-task learning — one network predicts favorite and purchase probabilities for an item.
  • Shared-bottom network — shared lower layers, separate task layers on top; the first milestone model.
  • Multi-gate Mixture of Experts (MMOE) — expert layer that lets each task learn its own representation; second milestone.
  • Module-name feature — tells the model which module a sample comes from.
  • Interaction-type weights — per-module sample weights for impressions, clicks, favorites and purchases.
Continue reading — it's free

Google or a one-time email code. No password.