Improving ETAs with multi-task models, deep learning, and probabilistic forecasts

Chi Zhang, Lewis Warne, Ziqi Jiang, Qingyang Xu, Hubert Jenq, Jianzhe Luo, Pradeep Varma12 Mar 2024

Problem

DoorDash shows an estimated time of arrival (ETA) on the home page, store page and checkout, for Dasher delivery, pickup and in-store grocery orders. Each case had its own model, which was costly to evolve and gave different ETAs at different stages of one order. The tree-based models also plateaued with more features or data, traded earliness against lateness, and handled rare use cases poorly. A point estimate hides how uncertain a delivery is (food preparation, Dasher acceptance, parking).

Example from the article: two deliveries both predicted at 20 minutes, one with a wide delivery-time distribution and one with a narrow one, need different handling.

Solution in brief

A two-layer system. The base layer is one multi-task deep model with a shared heavy foundation and a light head per ETA use case, and it outputs a probability distribution of delivery time. A separate decision layer turns that distribution into the ETA each business needs. The distribution is checked with PIT histograms (calibration) and CRPS (accuracy).

Questions this case answers
  • How to serve predictions for many related use cases with one model instead of a separate model per use case, while keeping outputs consistent across touchpoints
  • How to evaluate a model that predicts a full distribution of a long-tailed outcome rather than a point estimate
  • How to serve delivery-time estimates for many order types and app screens without a separate model for each?
  • How to predict a delivery-time distribution instead of a single number, so that uncertainty is visible?

Tools and methods

  • Multi-task model — shared foundation layer plus one lightweight task head per ETA use case.
  • Deep network with feature embeddings — replaces tree-based models.
  • Probabilistic base layer — predicts a delivery-time distribution.
  • Decision layer — multi-objective optimization per business on top of the distribution.
  • PIT histogram — checks calibration of the predicted distribution.
  • CRPS — one score for comparing probabilistic forecasts.
Continue reading — it's free

Google or a one-time email code. No password.