Problem
Etsy ranks sponsored listings in search with one multi-task model that predicts click-through rate (CTR) and post-click conversion rate (PCCVR). Both predictions feed ads ranking and auto-bidding, so they must be calibrated. Purchases are far sparser than clicks, and a click can be noise. The shared layers also cannot learn task-specific patterns, so improving one task can degrade another (the "seesaw phenomenon"). The team wanted a better purchase-intent signal without hurting clicks.
Example from the article: a user clicks an ad purely out of curiosity with no intention to buy.
Solution in brief
Replace the shared feed-forward layer with a multi-gate mixture of experts (MMoE): parallel expert networks plus one softmax gate per task. Add add-to-cart as an auxiliary task with its own gate and tower; it is trained but not served. Control expert usage with temperature scaling on the gate logits.
Questions this case answers
- How to improve prediction of a sparse, delayed outcome in a multi-task model when tasks interfere with each other and the shared layers cannot fit task-specific patterns
- How to pick auxiliary tasks for a sparse target task so that they add signal without negative transfer
- How to predict purchase intent better in an ads ranking model that already predicts clicks and purchases?
- How to stop one task from hurting another in a multi-task ranking model (the seesaw effect)?
Tools and methods
- Multi-gate mixture of experts (MMoE) — experts plus a per-task gate, to learn task-specific and shared patterns
- Auxiliary task (add-to-cart) — extra training signal for the sparse purchase task
- Deep & Cross Network (DCN) — explicit feature interactions before the experts; also used as an expert type
- Gate temperature scaling — makes each gate use several experts
- Platt scaling — calibrates the CTR and PCCVR towers
Google or a one-time email code. No password.