How We Cut Forecasting Time from Two Weeks to Five Minutes

Lyubov Levchenko25 May 2024

Problem

Avito's Forecasting team produces the baseline forecasts of business metrics (buyers, sellers, revenue and others) on which the company's quarterly budget is built. Finance and vertical teams then adjust them, and product initiatives are added on top. The baseline covers 5,500 series and is rebuilt every quarter. Done by hand (preprocessing, model choice, tuning, evaluation), it took one analyst about 4 weeks. It was also unclear whether RMSE, MAE and MAPE reflect what customers need from a forecast.

Example from the article: a Prophet forecast of the number of active "ducklings" ads, tuned by cross-validation on MAPE, predicted that such ads would soon disappear from the site.

Solution in brief

The team automated each stage. Outliers and level shifts become dummy regressors, tuning is scored by a metric on month-over-month changes (RMSE_MoM), and MSTL decomposition first shrinks the Prophet grid, then replaces Prophet as the forecasting model. With Nixtla's StatsForecast, 5,500 series are forecast in 5 minutes.

Questions this case answers
  • How to automate periodic forecasting of thousands of business metric series, choosing tuning by an error measure on month-over-month changes
  • How to turn outliers and level shifts in a series into regressors instead of deleting them before forecasting
  • How to automate quarterly forecasting of thousands of business metrics
  • How to speed up hyperparameter tuning of Prophet across many time series

Tools and methods

  • Dummy regressors — 1 during an anomalous period, 0 elsewhere
  • Modified Chow test — detects level shifts
  • Prophet — previous model
  • GridSearchCV + MAPE — first tuning scheme
  • RMSE_MoM — error of month-over-month changes, used for tuning and evaluation
  • MSTL (multiple seasonal-trend decomposition using Loess) — decomposition, and later the forecasting model
  • Nixtla StatsForecast — fast statistical models (MSTL, ARIMA, TBATS, ETS, Theta)
Continue reading — it's free

Google or a one-time email code. No password.