Problem
Operations teams watch business services and infrastructure through time series: requests per minute, tickets processed, CPU and disk utilization. A static threshold fires only after the metric has broken. Dynamic thresholds, built as an uncertainty interval around a forecast, fire earlier but still only when the actual value leaves the interval.
Example from the article: in the Sfera.IAS monitoring product, mean time to detection (MTTD) is 4 minutes for basic monitoring and 20 minutes for dynamic thresholds. A short-term forecast raises it to 2 hours.
Solution in brief
Build two forecasts of the same series: a long-term one, retrained rarely, and a 2-hour short-term one, retrained on fresh data and corrected by its recent error. If the short-term forecast leaves the uncertainty intervals or diverges from the long-term one, alert that a degradation is about to start. The base model is a seasonal regression trained without anomaly labels on CPUs.
Questions this case answers
- How to alert on a degradation before a univariate metric leaves its dynamic thresholds by comparing a short-term forecast retrained on fresh data with a long-term forecast
- How to monitor tens of thousands of time series in real time on CPU without labeled anomalies
- How to detect service degradation early with a two-horizon short-term forecast on univariate time series
- How to alert on an anomaly before the metric leaves its dynamic thresholds
Tools and methods
- Two-horizon forecast comparison — alert trigger
- Generalized additive regression (Stan) — base forecaster for seasonal series
- Forecast error correction — shifts the short-term forecast by its recent error
- VictoriaMetrics, ClickHouse — stores for real-time metrics and forecasts
- NeuralProphet — planned base model
Google or a one-time email code. No password.