Problem
DoorDash's homepage drives a large share of conversions, and consumers choose among more than 1,000 merchants on a small screen. The feed mixes carousels, single stores and items in a fixed order: carousels first, stores below. That worked with half a dozen carousels in 2020. By the third quarter of 2022 there were more than 30, so stores below them were rarely seen. A purely greedy ranker also keeps showing familiar merchants, reacts slowly to changed tastes and starves new merchants.
Example from the article: for a sushi and ramen fan, it is hard to compare one Japanese restaurant with a carousel of more than 10 popular Asian restaurants using separate per-type models.
Solution in brief
One deep ranking model, the universal ranker (UR), scores every entity by probability of conversion (pConv). A carousel is treated as an ordered list of stores and a store as an ordered list of items. An upper confidence bound (UCB) term, driven by how often the user has seen each entity, lifts rarely shown entities and lowers over-shown ones.
Questions this case answers
- How to rank heterogeneous content modules of different granularity in one feed with a single scoring model
- How to add controlled exploration to a learned ranker when each user sees only a few of thousands of candidates and has little feedback per item
- How to rank stores, carousels and items on one homepage with a single model
- How to keep a personalized feed from locking users into the same few merchants
Tools and methods
- Learning to rank (UR) — a Wide & Deep-style neural ranker in PyTorch that orders carousels, stores and items by pConv
- Upper confidence bound (UCB) — adds an uncertainty term to the expected conversion
- Elasticsearch — first-pass retrieval of up to 1,200 candidates
- Online experiments — tune the exploration coefficient
Google or a one-time email code. No password.