Product Reservoir Computing: Time-Series Computation with Multiplicative Neurons
Venue
2015 International Joint Conference on Neural Networks (IJCNN) (2015)
Publication Year
2015
Authors
Alireza Goudarzi, Alireza Shabani, Darko Stefanovic
BibTeX
Abstract
Echo state networks (ESN), a type of reservoir computing (RC) architecture, are
efficient and accurate artificial neural systems for time series processing and
learning. An ESN consists of a core of recurrent neural networks, called a
reservoir, with a small number of tunable parameters to generate a highdimensional
representation of an input, and a readout layer which is easily trained using
regression to produce a desired output from the reservoir states. Certain
computational tasks involve realtime calculation of high-order time correlations,
which requires nonlinear transformation either in the reservoir or the readout
layer. Traditional ESN employs a reservoir with sigmoid or tanh function neurons.
In contrast, some types of biological neurons obey response curves that can be
described as a product unit rather than a sum and threshold. Inspired by this class
of neurons, we introduce a RC architecture with a reservoir of product nodes for
time series computation. We find that the product RC shows many properties of
standard ESN such as short-term memory and nonlinear capacity. On standard
benchmarks for chaotic prediction tasks, the product RC maintains the performance
of a standard nonlinear ESN while being more amenable to mathematical analysis. Our
study provides evidence that such networks are powerful in highly nonlinear tasks
owing to highorder statistics generated by the recurrent product node reservoir.
