Autoencoder Based Residual Deep Networks for Robust Regression Prediction and Spatiotemporal Estimation
This work addresses the challenge of robust regression prediction for domains with small datasets, such as environmental monitoring, though it appears incremental as it builds on existing residual network concepts.
The authors tackled the problem of deep learning's limited applicability in regression tasks with small sample sizes and potential accuracy degradation from many hidden layers, by proposing an autoencoder-based residual deep network that achieved cutting-edge accuracy and efficiency in spatiotemporal estimation tasks.
To have a superior generalization, a deep learning neural network often involves a large size of training sample. With increase of hidden layers in order to increase learning ability, neural network has potential degradation in accuracy. Both could seriously limit applicability of deep learning in some domains particularly involving predictions of continuous variables with a small size of samples. Inspired by residual convolutional neural network in computer vision and recent findings of crucial shortcuts in the brains in neuroscience, we propose an autoencoder-based residual deep network for robust prediction. In a nested way, we leverage shortcut connections to implement residual mapping with a balanced structure for efficient propagation of error signals. The novel method is demonstrated by multiple datasets, imputation of high spatiotemporal resolution non-randomness missing values of aerosol optical depth, and spatiotemporal estimation of fine particulate matter <2.5 μm, achieving the cutting edge of accuracy and efficiency. Our approach is also a general-purpose regression learner to be applicable in diverse domains.