elib
DLR-Header
DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Breaking symmetries of the reservoir equations in echo state networks

herteux, Joschka and Räth, Christoph (2020) Breaking symmetries of the reservoir equations in echo state networks. Chaos, 30 (12), p. 123142. American Institute of Physics (AIP). doi: 10.1063/5.0028993. ISSN 1054-1500.

[img] PDF - Published version
4MB

Official URL: https://aip.scitation.org/doi/10.1063/5.0028993

Abstract

Reservoir computing has repeatedly been shown to be extremely successful in the prediction of nonlinear time-series. However, there is no complete understanding of the proper design of a reservoir yet. We find that the simplest popular setup has a harmful symmetry, which leads to the prediction of what we call mirror-attractor. We prove this analytically. Similar problems can arise in a general context, and we use them to explain the success or failure of some designs. The symmetry is a direct consequence of the hyperbolic tangent activation function. Furthermore, four ways to break the symmetry are compared numerically: A bias in the output, a shift in the input, a quadratic term in the readout, and a mixture of even and odd activation functions. First, we test their susceptibility to the mirror-attractor. Second, we evaluate their performance on the task of predicting Lorenz data with the mean shifted to zero. The short-time prediction is measured with the forecast horizon while the largest Lyapunov exponent and the correlation dimension are used to represent the climate. Finally, the same analysis is repeated on a combined dataset of the Lorenz attractor and the Halvorsen attractor, which we designed to reveal potential problems with symmetry. We find that all methods except the output bias are able to fully break the symmetry with input shift and quadratic readout performing the best overall.

Item URL in elib:https://elib.dlr.de/139952/
Document Type:Article
Title:Breaking symmetries of the reservoir equations in echo state networks
Authors:
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
herteux, JoschkaUNSPECIFIEDUNSPECIFIEDUNSPECIFIED
Räth, ChristophUNSPECIFIEDUNSPECIFIEDUNSPECIFIED
Date:December 2020
Journal or Publication Title:Chaos
Refereed publication:Yes
Open Access:Yes
Gold Open Access:No
In SCOPUS:Yes
In ISI Web of Science:Yes
Volume:30
DOI:10.1063/5.0028993
Page Range:p. 123142
Publisher:American Institute of Physics (AIP)
ISSN:1054-1500
Status:Published
Keywords:artificial intelligence, machine learning, reservoir computing, prediction, time series, chaotic systems, symmetries
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Research under Space Conditions
DLR - Research area:Raumfahrt
DLR - Program:R FR - Research under Space Conditions
DLR - Research theme (Project):R - Komplexe Plasmen / Data analysis (old)
Location: Oberpfaffenhofen
Institutes and Institutions:Institute of Materials Physics in Space > Research Group Complex Plasma
Deposited By: Räth, Christoph
Deposited On:08 Jan 2021 12:11
Last Modified:24 Oct 2023 11:24

Repository Staff Only: item control page

Browse
Search
Help & Contact
Information
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.