Unified Training of Universal Time Series Forecasting Transformers

Deep learning for time series forecasting traditionally uses a one-model-per-dataset approach, limiting potential advancements.
Universal forecasting introduces the idea of pre-training a single Large Time Series Model on a vast collection of datasets for diverse tasks.
Challenges in creating such a model include: cross-frequency learning, handling multivariate series with arbitrary variates, and varying distributional properties of large-scale data.
To overcome these challenges, novel enhancements to the time series Transformer architecture are introduced, creating the Masked EncOder-based UnIveRsAl TIme Series Forecasting Transformer (MOIRAI).
MOIRAI is trained on the Large-scale Open Time Series Archive (LOTSA), which contains over 27 billion observations across nine domains.
MOIRAI demonstrates competitive or superior performance as a zero-shot forecaster compared to full-shot models.

Deeplearning.fr