Reservoir Computing and Its Application in Prediction Tasks

Reservoir Computing(RC) is a special framework of RNNs. As in the classical RNN framework, the RC model is composed of three parts: the input layer, the inner (reservoir) layer, and the readout layer. The merit of RC is that only the readout layer needs to be trained whereas the input and the inner weights are fixed all the time. As a representative of RC models, Echo State Network (ESN) has been widely studied for temporal data prediction and classification. Our current works focus on improving the prediction performance of ESN-based models.

A simple schematic diagram of the echo state network

Temporal Feature Extraction with Deep Reservoir Computing

With the development of deep learning, the concept of stacking architecture has been introduced into RC models. Deep RC was first introduced in where a deep RC model called DeepESN with stacking multiple ESNs was proposed. The merit of DeepESN inherits that of ESN which only the readout part can be trained. Based on DeepESN, we proposed two novel models that can use non-sequential time spans to afford more information. Our next goal is to continue exploring the theoretical basis of our proposed two Deep RC models.

A simple schematic diagram of the Deep Multi-Span Echo State Network