Time Series Prediction by Transformer

Abstract

Time series prediction is important regardless of whether it is univariate or multivariate, and univariate ARIMA to multivariate autoregressive model VAR etc. have been used as machine learning techniques for a long time, and deep learning such as RNN and CNN are used for time-series recently. In recent years, LSTNet and MTNet, which are hybrid models of before models,have emerged to further improve the accuracy of multivariate time series prediction. In other hand, the natural language processing such as Transformer has achieved great progress by achieving state-of-the-art in sequence prediction of natural language, and later evolving into BERT, which enables transfer learning. The purpose of this research is to evaluate the possibility of applying the natural language processing technology to time-series prediction. We designed a temperature prediction model based on Transformer, and found that Transformer can provide better accuracy in time series prediction by data assimilation such as weather ensemble forecasting.