Using Gamma Distribution to Improve Long-Tail Event Predictions

Pratik Parekh, Zhe Jia6 Apr 2022

Problem

A delivery platform promises customers an estimated arrival time (ETA). A late delivery ruins the order, and an inflated estimate makes people skip the order or miss the delivery. Delivery time is hard to predict because it depends on preparation time, traffic, courier availability and the time to find parking and reach the door. These rare, costly long-tail events are where predictions are far off from actual times.

Example from the article: DoorDash already tweaked its loss function and added real-time signals, and then wanted a further gain on long-tail predictions of delivery duration.

Solution in brief

Fit a distribution to the real target values, then give each training sample a weight inversely proportional to its density, so rare long deliveries count more. Split the samples at the mode of the distribution and tune the weights on the two sides separately, one for earliness and one for lateness.

Questions this case answers
  • How to make a regression model for a right-skewed positive target accurate on the rare extreme values while controlling errors on both sides of the typical value
  • How to make a delivery-time model predict rare, very late orders better without ruining predictions for typical orders?
  • How to choose a distribution that matches the target variable and check the fit statistically?
  • How to weight training samples so that both early and late errors can be tuned to business cost?

Tools and methods

  • Gamma distribution fit — model the right-skewed, always-positive delivery time and derive sample weights from its density.
  • Kolmogorov-Smirnov (K-S) test — compare candidate distributions (normal, skew normal, log-normal, gamma) with the empirical one.
  • Inverse-density sample weights, split at the mode — raise the loss contribution of long-tail samples, with separate left and right weights.
  • Asymmetric Mean Squared Error (AMSE) — the earlier loss used as the baseline.
Continue reading — it's free

Google or a one-time email code. No password.