TIME-MOE : Time series

Publié le par

BILLION-SCALE TIME SERIES FOUNDATION MODELS From Princeton WITH MIXTURE OF EXPERTS

TIME-MOE is a scalable and unified architecture designed for pre-training large, capable forecasting foundation models while reducing inference costs. It addresses the limitations of current pre-trained time series models, which are often limited in scale and operate at high costs.

Key Features

  • Sparse Mixture-of-Experts (MoE) Design: Enhances computational efficiency by activating only a subset of networks for each prediction.
  • Scalability: Allows for effective scaling without a corresponding increase in inference costs.
  • Flexibility: Supports flexible forecasting horizons with varying input context lengths.

Architecture

  • Decoder-only transformer models
  • Operates in an autoregressive manner
  • Family of models scaling up to 2.4 billion parameters

Training Data

  • Pre-trained on Time-300B dataset
  • Spans over 9 domains
  • Encompasses over 300 billion time points

Performance

  • Achieves significantly improved forecasting precision
  • Outperforms dense models with equivalent computation budgets or activated parameters

Applications

Positioned as a state-of-the-art solution for real-world time series forecasting challenges, offering superior capability, efficiency, and flexibility.

https://arxiv.org/pdf/2409.16040

Code:

https://github.com/Time-MoE/Time-MoE

Scalable. Interactive. Interpretable Data Science

Publié le par

Safe, interpretable, trustworthy AI, through interactive intelligent visualization, with applications in adversarial machine learning (protecting AI from harm and doing harm), scalable discoveries of deep learning models, and inclusive AI for everyone.

A Must.

https://poloclub.github.io/

Transformer Explainer

Learn How Transformer Models Work with Interactive Visualization

https://poloclub.github.io/transformer-explainer/

    Diffusion Explainer

    Learn how Stable Diffusion transforms your text prompt into image.

    https://poloclub.github.io/diffusion-explainer/

    User-Centric RAG

    Publié le par

    Transforming RAG with LlamaIndex Multi-Agent System and Qdrant

    Retrieval-Augmented Generation (RAG) models have evolved significantly over time. Initially, traditional RAG systems faced numerous limitations. However, with advancements in the field, we have seen the emergence of more sophisticated RAG applications. Techniques such as Self-RAG, Hybrid Search RAG, experimenting with different prompting and chunking strategies, and the evolution of Agentic RAG have addressed many of the initial limitations.

    https://medium.com/@pavannagula76/user-centric-rag-transforming-rag-with-llamaindex-multi-agent-system-and-qdrant-cf3c32cfe6f3

    PDF-Extract-Kit

    Publié le par

    PDF-Extract-Kit, a comprehensive toolkit for high-quality PDF content extraction, including layout detection, formula detection, formula recognition, and OCR.

    PDF documents contain a wealth of knowledge, yet extracting high-quality content from PDFs is not an easy task. To address this, we have broken down the task of PDF content extraction into several components:

    • Layout Detection: Using the LayoutLMv3model for region detection, such as images, tables, titles, text, etc.;
    • Formula Detection: Using YOLOv8 for detecting formulas, including inline formulas and isolated formulas;
    • Formula Recognition: Using UniMERNet for formula recognition;
    • Table Recognition: Using StructEqTable for table recognition;
    • Optical Character Recognition: Using PaddleOCR for text recognition;

    https://github.com/opendatalab/PDF-Extract-Kit

    https://www.perplexity.ai/search/look-at-this-github-https-gith-8ZVtYO.2SA6_q5Vg.VXy.g

    Self-RAG

    Publié le par

    Self-RAG is another form of Retrieval Augmented Generation (RAG). Unlike other RAG retrieval strategies, it doesn’t enhance a specific module within the RAG process. Instead, it optimizes various modules within the RAG framework to improve the overall RAG process. If you’re unfamiliar with Self-RAG or have only heard its name, join me today to understand the implementation principles of Self-RAG and better grasp its details through code.

    https://ai.gopubby.com/advanced-rag-retrieval-strategies-self-rag-3e9a4cd422a1

    https://llamahub.ai/l/llama-packs/llama-index-packs-self-rag?from=

    Rag techniques notebook

    Publié le par

    This repository showcases various advanced techniques for Retrieval-Augmented Generation (RAG) systems. RAG systems combine information retrieval with generative models to provide accurate and contextually rich responses.

    https://github.com/NirDiamant/RAG_Techniques

    https://github.com/NirDiamant/RAG_Techniques/tree/main/all_rag_techniques

    https://github.com/NirDiamant/RAG_Techniques/tree/main/all_rag_techniques