Contractive De-noising Auto-encoder
This work addresses feature robustness in neural networks for researchers in machine learning, but it is incremental as it combines existing techniques.
The paper tackled the problem of learning robust features in auto-encoders by combining de-noising and contractive auto-encoders into a contractive de-noising auto-encoder (CDAE), which improved performance on the MNIST dataset compared to individual methods.
Auto-encoder is a special kind of neural network based on reconstruction. De-noising auto-encoder (DAE) is an improved auto-encoder which is robust to the input by corrupting the original data first and then reconstructing the original input by minimizing the reconstruction error function. And contractive auto-encoder (CAE) is another kind of improved auto-encoder to learn robust feature by introducing the Frobenius norm of the Jacobean matrix of the learned feature with respect to the original input. In this paper, we combine de-noising auto-encoder and contractive auto- encoder, and propose another improved auto-encoder, contractive de-noising auto- encoder (CDAE), which is robust to both the original input and the learned feature. We stack CDAE to extract more abstract features and apply SVM for classification. The experiment result on benchmark dataset MNIST shows that our proposed CDAE performed better than both DAE and CAE, proving the effective of our method.