QuitoBench: A High-Quality Open Time Series Forecasting Benchmark
Time series forecasting is critical across finance, healthcare, and cloud computing, yet progress is constrained by a fundamental bottleneck: the scarcity of large-scale, high-quality benchmarks. To address this gap, we introduce QuitoBench, a regime-balanced benchmark for time series forecasting with coverage across eight trendtimesseasonalitytimesforecastability (TSF) regimes, designed to capture forecasting-relevant properties rather than application-defined domain labels. The benchmark is built upon Quito, a billion-scale time series corpus of application traffic from Alipay spanning nine business domains. Benchmarking 10 models from deep learning, foundation models, and statistical baselines across 232,200 evaluation instances, we report four key findings: (i) a context-length crossover where deep learning models lead at short context (L=96) but foundation models dominate at long context (L ge 576); (ii) forecastability is the dominant difficulty driver, producing a 3.64 times MAE gap across regimes; (iii) deep learning models match or surpass foundation models at 59 times fewer parameters; and (iv) scaling the amount of training data provides substantially greater benefit than scaling model size for both model families. These findings are validated by strong cross-benchmark and cross-metric consistency. Our open-source release enables reproducible, regime-aware evaluation for time series forecasting research.
69ccf5291f6782218d973dec
Siqiao Xue
69ccf5291f6782218d973ded
Zhaoyang Zhu
69ccf5291f6782218d973dee
Wei Zhang
69ccf5291f6782218d973def
Rongyao Cai
69ccf5291f6782218d973df0
Rui Wang
69ccf5291f6782218d973df1
Yixiang Mu
69ccf5291f6782218d973df2
Fan Zhou
69ccf5291f6782218d973df3
Jianguo Li
69ccf5291f6782218d973df4
Peng Di
69ccf5291f6782218d973df5
Hang Yu
QuitoBench: A High-Quality Open Time Series Forecasting Benchmark
Time series forecasting is critical across finance, healthcare, and cloud computing, yet progress is constrained by a fundamental bottleneck: the scarcity of large-scale, high-quality benchmarks. To address this gap, we introduce QuitoBench, a regime-balanced benchmark for time series forecasting with coverage across eight trendtimesseasonalitytimesforecastability (TSF) regimes, designed to capture forecasting-relevant properties rather than application-defined domain labels. The benchmark is built upon Quito, a billion-scale time series corpus of application traffic from Alipay spanning nine business domains. Benchmarking 10 models from deep learning, foundation models, and statistical baselines across 232,200 evaluation instances, we report four key findings: (i) a context-length crossover where deep learning models lead at short context (L=96) but foundation models dominate at long context (L ge 576); (ii) forecastability is the dominant difficulty driver, producing a 3.64 times MAE gap across regimes; (iii) deep learning models match or surpass foundation models at 59 times fewer parameters; and (iv) scaling the amount of training data provides substantially greater benefit than scaling model size for both model families. These findings are validated by strong cross-benchmark and cross-metric consistency. Our open-source release enables reproducible, regime-aware evaluation for time series forecasting research.
QuitoBench addresses the lack of large-scale time series benchmarks by introducing a regime-balanced dataset with eight TSF regimes, revealing that foundation models outperform deep learning at long contexts while scaling data provides greater benefits than scaling model size.
time series forecasting deep learning models foundation models context-length crossover forecastability TSF regimes regime-balanced benchmark billion-scale time series corpus training data scaling model size scaling