One Fits All: Power General Time Series Analysis by Pretrained LM (NeurIPS 2023 Spotlight)

Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, Rong Jin, "One Fits All: Power General Time Series Analysis by Pretrained LM,", NeurIPS, 2023. [paper]

The main challenge that blocks the development of pre-trained model for time series analysis is the lack of a large amount of data for training. In this work, we address this challenge by leveraging language or CV models, pre-trained from billions of tokens, for time series analysis. Specifically, we refrain from altering the self-attention and feedforward layers of the residual blocks in the pre-trained language or image model.

General Time Series Tasks

The proposed method outperforms other models on most tasks, including long-term forecasting, short-term forecasting, classification, anomaly detection, imputation, and few-shot leanring, zero-short learning.

Get Start

• Install Python>=3.8, PyTorch 1.8.1.
• Follow the instructions provided in the respective task folder.

Citation

If you find this repo useful, please cite our paper.

@inproceedings{zhou2023onefitsall,
  title={{One Fits All}: Power General Time Series Analysis by Pretrained LM},
  author={Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, Rong Jin},
  booktitle={NeurIPS},
  year={2023}
}

Further Reading

Survey on Transformers in Time Series:

Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. "Transformers in time series: A survey.", IJCAI, 2023. [paper]

Contact

If you have any question or want to use the code, please contact tian.zt@alibaba-inc.com or niupeisong.nps@alibaba-inc.com .

Acknowledgement

We appreciate the following github repos a lot for their valuable code base or datasets:

https://github.com/DAMO-DI-ML/ICML2022-FEDformer

https://github.com/thuml/Time-Series-Library

https://github.com/gzerveas/mvts_transformer