Training Auto-encoders Effectively via Eliminating Task-irrelevant Input Variables
This addresses the issue of poor generalization in deep networks for researchers and practitioners, but it is incremental as it builds on existing auto-encoder frameworks.
The paper tackles the problem of task-irrelevant information degrading auto-encoder performance by proposing a method to drop such variables, resulting in significantly improved performance on three challenging datasets.
Auto-encoders are often used as building blocks of deep network classifier to learn feature extractors, but task-irrelevant information in the input data may lead to bad extractors and result in poor generalization performance of the network. In this paper,via dropping the task-irrelevant input variables the performance of auto-encoders can be obviously improved .Specifically, an importance-based variable selection method is proposed to aim at finding the task-irrelevant input variables and dropping them.It firstly estimates importance of each variable,and then drops the variables with importance value lower than a threshold. In order to obtain better performance, the method can be employed for each layer of stacked auto-encoders. Experimental results show that when combined with our method the stacked denoising auto-encoders achieves significantly improved performance on three challenging datasets.