Predicting chaotic time series by means of reservoir computing

Abstract

It has been demonstrated that in the realm of complex systems not only exact predic-tions of multivariate time series with large time horizons become possible but also the longterm behavior of the underlying dynamical system (its climate) can well be reproducedusing machine learning techniques. This is achieved by using reservoir computing (RC), which represents a special kind ofrecurrent neural networks (RNN). The core of the model is a network called reservoir,which is a complex network with loops. Input data are fed into the nodes of the reservoir,which are connected according to a predefined network topology (mostly random net-works). Only the weights of the linear output layer transforming the reservoir response tooutput variables are subject to optimization via linear regression. This makes the learningextremely fast and omits the vanishing gradient problem of other RNNs.Here, we investigate the question of statistical stability of short and long term predictionsand find that the ability to exactly forecast the correct trajectory as well as the reconstruc-tion of the long-term climate measured by the correlation dimension and largest Lyapunovexponent strongly varies among different realizations of the same reservoir setup [1]. Thus,special care must be taken in selecting the good predictions.To improve upon the statistical robustness of the prediction results we tested differentnetwork topologies, namely (random) Erd ̈os Renyi, small world and scale free networksfor the reservoir. While the small world reservoir showed slightly better results for theLorenz system than a random network, the scale-free network performed worse, where thedifference to the other two network types is much pronounced for the Roessler system [1]. In-depth studies reveal that the nodes which contribute most to the output signal are notthose, which have most connections in the network. Thus scale-free networks with somehighly connected nodes do not represent the suitable topology for this kind of predictiontask. Furthermore, we demonstrate that a controlled node removal and a suitable chosenweighting function significantly increase the prediction performance - even with a muchsmaller reservoir [2]. Studying and interpreting these results in detail will give new insights about the essentialrequirements for the emergence of complex behavior in nonlinear dynamical systems. [1] A. Haluszczynski & C. Räth, Chaos 29, 103143 (2019) [2] A. Haluszczynski, J. Aumeier, J. Herteux & C. Räth, Chaos 30, 063136 (2020)