Minimal Reservoir Computing for Weakly and Strongly Nonlinear Time Series

Abstract

The traditional setup of reservoir computing (RC) uses random matrices to define the underlying network and the input layer that transforms the input data to input signals in the reservoir. Here, we show that a few simple modifications, which eliminate randomness and minimize computational resources, lead to significant and robust improvements in short- and long-term predictive performance. We first introduce block-diagonal reservoirs, which implies that a reservoir can be composed of multiple smaller reservoirs. In a second step the non-linear activation function at the nodes can be dispensed with if the non-linear step in the analysis chain is shifted to the output layer. This means that not only the reservoir echo but also its higher powers are used to optimize the output weights. The input weights are determined according to well-defined rules. Any random initialization has thus been eliminated in this approach. By varying the remaining four hyperparameters, it is now possible to systematically investigate the transition from a linear, disjunct mapping of the input data to the output data to a combined nonlinear one. It is investigated how the prediction performance varies during this transition. The results are interpreted with respect to minimal requirements of RC for a proper prediction of weakly and strongly nonlinear time series. These results can guide the way towards explainable RC, where the prediction process becomes nearly fully transparent.