Forecast and chill: Zero-Shot Foundation Models for Time Series

Ksenia Kuvshinova25 May 2024

Problem

Teams that forecast many regular series often meet series with only dozens of points. Boosting needs a few hundred points and a transformer needs more, so no model can be trained for such series, yet a forecast is still required. Even with enough data, a baseline per series means extra engineering to train on real data.

Example from the article: a few points of a short series are fed into a pretrained model as context, and it returns a baseline forecast without retraining. Many such series can be fed in at once.

Solution in brief

Use a pretrained time series foundation model in zero-shot mode: pass the history of length L as context and read the forecast for horizon H. Open models such as Chronos, MOMENT and Moirai are ready to try. If a few hundred points exist, fine-tune on them (few-shot) and size the budget in seasonal periods, not in percent of the train length.

Questions this case answers
  • How to forecast many regular series that have too few points to train a model on, by reusing a pretrained sequence model without per-series training
  • How to benchmark pretrained forecasting models fairly across few-shot setups by counting seasonal periods rather than percent of the training length
  • How to forecast regular time series that are too short to train a model on
  • How to get a quick forecast baseline for many series without training a model per series

Tools and methods

  • Chronos — open model that scales the series, quantizes values into tokens and predicts the next token
  • MOMENT, Moirai, Lag-Llama, TinyTimeMixers, TOTEM — other open foundation models compared in the talk
  • TimesFM, TimeGPT-1 — closed models in the same comparison
  • TSUFormer — the speaker's encoder, pretrained on synthetic data, for forecasting, trend and seasonality extraction
  • Few-shot benchmark — fine-tuning on a budget of points, from zero-shot to full supervised
Continue reading — it's free

Google or a one-time email code. No password.