DropIn: Making Reservoir Computing Neural Networks Robust to Missing Inputs by Dropout
This addresses the issue of unreliable or disconnected input sources in applications like pervasive wireless sensor networks and ambient intelligence, representing an incremental improvement by adapting dropout regularization for robustness.
The paper tackles the problem of making Reservoir Computing neural networks robust to missing input features at prediction time, and the result is that the proposed DropIn methodology maintains predictive performance comparable to a model without missing features even when 20%-50% of inputs are unavailable.
The paper presents a novel, principled approach to train recurrent neural networks from the Reservoir Computing family that are robust to missing part of the input features at prediction time. By building on the ensembling properties of Dropout regularization, we propose a methodology, named DropIn, which efficiently trains a neural model as a committee machine of subnetworks, each capable of predicting with a subset of the original input features. We discuss the application of the DropIn methodology in the context of Reservoir Computing models and targeting applications characterized by input sources that are unreliable or prone to be disconnected, such as in pervasive wireless sensor networks and ambient intelligence. We provide an experimental assessment using real-world data from such application domains, showing how the Dropin methodology allows to maintain predictive performances comparable to those of a model without missing features, even when 20\%-50\% of the inputs are not available.