Challenges in Time Series Forecasting Models

The biggest challenge in building a pre-trained model for time series is finding high-quality and diverse data. This difficulty is at the core of developing effective forecasting models.

Main Approaches

Two primary approaches are used to build a fundamental forecasting model:

1. Adapting an LLM: This method involves repurposing a pre-trained language model like GPT-4 or Llama by adapting it to time series tasks.
2. Building from Scratch: This approach involves creating a vast time series dataset to pre-train a model, hoping it will generalize to new data.

Results and Limitations

The second approach has proven more effective, as evidenced by models such as MOIRAI, TimesFM, and TTM. However, these models follow scaling laws, and their performance heavily depends on the availability of extensive time series data, which brings us back to the initial challenge.

Innovation: Using Images

Faced with these limitations, an innovative approach was explored: using a different modality, namely images. Although counterintuitive, this method has produced groundbreaking results, opening new perspectives in the field of time series forecasting.

VisionTS: A New Paradigm

VisionTS represents a novel approach that leverages the power of image-based models for time series forecasting. This method transforms time series data into images, allowing the use of advanced computer vision techniques to predict future values.

Advantages of Image-Based Forecasting

Using images for time series forecasting offers several advantages:

• Access to a vast pool of pre-trained image models
• Ability to capture complex patterns and relationships in data
• Potential for transfer learning from diverse image datasets

Future Implications

The success of VisionTS suggests a promising direction for future research in time series forecasting. It demonstrates the potential of cross-modal learning and opens up new possibilities for improving prediction accuracy and generalization in various domains.

paper:

https://arxiv.org/pdf/2408.17253

Code:

https://github.com/Keytoyze/VisionTS

BILLION-SCALE TIME SERIES FOUNDATION MODELS From Princeton WITH MIXTURE OF EXPERTS

TIME-MOE is a scalable and unified architecture designed for pre-training large, capable forecasting foundation models while reducing inference costs. It addresses the limitations of current pre-trained time series models, which are often limited in scale and operate at high costs.

Key Features

• Sparse Mixture-of-Experts (MoE) Design: Enhances computational efficiency by activating only a subset of networks for each prediction.
• Scalability: Allows for effective scaling without a corresponding increase in inference costs.
• Flexibility: Supports flexible forecasting horizons with varying input context lengths.

Architecture

• Decoder-only transformer models
• Operates in an autoregressive manner
• Family of models scaling up to 2.4 billion parameters

Training Data

• Pre-trained on Time-300B dataset
• Spans over 9 domains
• Encompasses over 300 billion time points

Performance

• Achieves significantly improved forecasting precision
• Outperforms dense models with equivalent computation budgets or activated parameters

Applications

Positioned as a state-of-the-art solution for real-world time series forecasting challenges, offering superior capability, efficiency, and flexibility.

https://arxiv.org/pdf/2409.16040

Code:

https://github.com/Time-MoE/Time-MoE