Problem
Avito sellers pay to promote a listing. Before paying, they choose a budget and a duration and see a forecast of the extra views as a range, for example 5–8. After the promotion system changed (a new auction in home-page recommendations), the old forecast hit the range less often, and users complained. The team also did not know how forecast quality affects purchases, average check and churn.
Example from the article: a forecast of 5–8 views and an actual result of 3 looks like a deception and goes to support. A result of 10 is fine for the user. A forecast of 5–100 is always "right" but tells the user nothing.
Solution in brief
The team defined four forecast metrics (accuracy, over-forecast share, under-forecast share, interval width) and fixed the width at the production level while maximizing accuracy. The forecast is a point prediction times left and right coefficients. The point prediction comes from a cascade: extrapolation from the listing's previous promotion, then CatBoost, then a lookup table.
Questions this case answers
- How to forecast the outcome of a paid promotion for a given budget and duration and show it as a range that the real result falls inside often enough
- How to compare forecasting models when interval accuracy and interval width pull in opposite directions and the forecast must work for new items
- How to forecast the number of views a paid promotion will bring for a given budget and duration
- How to measure the quality of an interval forecast shown to users
Tools and methods
- CatBoostRegressor with a monotonicity constraint on budget: point forecast for items without history
- Historical liquidity — scale the views of the previous promotion by the budget ratio
- Interval coefficients — left and right multipliers on the point forecast, tunable without retraining
- A/B test — online check of revenue and forecast metrics
Google or a one-time email code. No password.