Problem
Wix forecasts collections income far ahead, and stakeholders plan on it. Patterns drift with seasons, markets and one-off events, so the model needs frequent retraining. Transformer-based models train slowly. Their long-horizon forecasts change a lot between retrains even when accuracy looks stable. A simulation in the post shows that small weekly changes can add up to a $15M week-on-week swing in the forecast, which makes planning hard.
Example from the article: collections that depend on planned sale days and new-user sign-up trends, with weekly retraining.
Solution in brief
Wix uses TiDE, an MLP-based forecaster that trains much faster than transformers and accepts external covariates. Retraining is weekly and the published forecast is the average of the last 4 models. The last part of the training set doubles as validation, and hyperparameters are tuned with Optuna over rolling cross-validation. The team reports a 40% improvement in the collections income forecast.
Questions this case answers
- How to make a periodically retrained long-horizon forecast stable enough that stakeholders keep trusting it while it still reacts to new trends
- How to tune and validate a neural time-series model when rolling-origin cross-validation is too slow and the latest data is reserved for validation
- How to keep a deep learning forecast of a business metric current without retraining costs getting out of hand?
- How to stop long-horizon forecasts from jumping every time the model is retrained?
Tools and methods
- TiDE (Time-series Dense Encoder) — encoder-decoder forecaster built from MLPs, supports exogenous covariates, fast to retrain
- Ensemble of the last 4 retrained models — averaging smooths forecast revisions and may add accuracy
- Train-tail validation — the final part of the training set is also the validation set
- Rolling cross-validation (forward chaining) — monthly backtests across a year to score a hyperparameter set
- Optuna — Bayesian hyperparameter search that cuts the number of training runs
Google or a one-time email code. No password.