Problem
DoorDash forecasts supply (Dashers, the delivery drivers) and demand for thousands of small ZIP-code groups, called starting points. The forecasts drive Dasher acquisition and extra pay when supply looks low. A gradient boosting machine (GBM) handles ordinary days well, but its accuracy drops on holidays. Each holiday occurs once a year, and a tree that avoids overfitting puts the extreme holiday days into the same leaf and predicts their average. Bad holiday forecasts force ad-hoc manual fixes by operations and ML teams.
Example from the article: if order volume falls 50% on Thanksgiving, and the holidays in one leaf average to -35%, the forecast errs by +15% for Thanksgiving and by -10% for the Fourth of July.
Solution in brief
Estimate a holiday multiplier per starting point and daypart with a linear regression on holiday dummies. Divide the history by these multipliers to get "holiday-free" series, train the GBM on them, then apply the multipliers to the GBM forecast. Because the series is corrected rather than split, the most recent lags stay available after a holiday.
Questions this case answers
- How to forecast a time series around rare, high-variance calendar events when a tree-based model has only one observation per event per year
- How to justify shipping a model change when the only interference-free experiment design is too underpowered to measure the effect
- How to keep demand and supply forecasts accurate on holidays when a tree-based model sees each holiday once a year?
- How to correct for holiday effects without losing the most recent lags of the series?
Tools and methods
- Gradient boosting machine (GBM) — base forecaster, trained on holiday-free data
- Linear regression with holiday dummies — one per starting point-daypart pair; coefficients are the multipliers
- Apache Spark — fits the thousands of regressions in parallel
- Backtesting — estimates the quality impact on past holidays when an experiment is too weak
Google or a one-time email code. No password.