Using a Multi-Armed Bandit with Thompson Sampling to Identify Responsive Dashers

Arjun Sharma15 Mar 2022

Problem

DoorDash needs enough couriers (Dashers) online to deliver orders on time. When supply in a location runs short, it sends push messages asking Dashers to sign on. Recipients were picked at random among recently active Dashers. That does not overload anyone, but it does not raise the share of messages that lead to a sign-on, and messages to uninterested Dashers hurt their experience.

The team wanted to rank Dashers by willingness to dash now, avoid messaging the uninterested, and keep discovering newly responsive Dashers so the same few are not messaged every time.

Example from the article: a Dasher who is a college student responds during breaks and is unavailable once classes resume, so a model trained on old data keeps targeting the wrong people.

Solution in brief

Each Dasher gets a Beta distribution over the probability of signing on after a message. Before every send, the system samples one value per Dasher, ranks by the sample and messages the top Dashers above a threshold. Every outcome updates that Dasher's distribution, so exploration shrinks as certainty grows and the ranking follows changing behavior.

Questions this case answers
  • How to select which individuals to contact in a limited-capacity outreach channel when their response propensity drifts over time and the policy must keep exploring
  • How to measure the effect of a targeting policy on an aggregate outcome when individual units share one market and cannot be randomized separately
  • How to choose which couriers to send a push message to when supply is short, without spamming those who will not respond?
  • How to keep finding newly responsive recipients while mostly messaging the ones known to respond?

Tools and methods

  • Thompson sampling with Beta priors — picks recipients by sampling each Dasher's response probability and updating it after each message
  • Weight decay on past observations — keeps the prior fresh when behavior changes
  • Switchback experiment — compares the bandit with random selection at the level of a whole location
Continue reading — it's free

Google or a one-time email code. No password.