Tailored minimal reservoir computing: Connecting nonlinearities in the input data with nonlinearities in the reservoir

Abstract

The traditional setup of reservoir computing (RC) for predicting time series uses random matrices to define the underlying network and the input layer. Here, we show that a few modifications, which eliminate randomness and minimize computational resources and data requirements, lead to significant and robust improvements in short- and long-term predictive performance. We introduce block-diagonal reservoirs, which implies that a reservoir can be composed of multiple smaller reservoirs. Further, the non-linear activation function at the nodes can be dispensed with if the non-linear step in the analysis chain is shifted to the output layer. The input weights are determined according to well-defined rules. Any random initialization has thus been eliminated. By varying the remaining four hyperparameters, it is now possible to systematically investigate the transition from a linear, disjunct mapping of the input data to the output data to a combined nonlinear one. It is further demonstrated that there is a connection between the nonlinearities in the input data and the nonlinearities in the reservoir such that the best prediction results are obtained when both nonlinearities match. It becomes thus possible to define an optimally tailored setup for minimal RC for data sets with given nonlinearities.