Learning Robust Representations of Text
This addresses robustness issues in NLP models for tasks like sentiment analysis, but it is incremental as it adapts ideas from computer vision.
The paper tackles the problem of deep neural networks being sensitive to noise and adversarial attacks in language processing by introducing a regularization-based method to learn more robust models. The result shows superior performance over noisy inputs and out-of-domain data compared to baseline and dropout methods on sentiment datasets.
Deep neural networks have achieved remarkable results across many language processing tasks, however these methods are highly sensitive to noise and adversarial attacks. We present a regularization based method for limiting network sensitivity to its inputs, inspired by ideas from computer vision, thus learning models that are more robust. Empirical evaluation over a range of sentiment datasets with a convolutional neural network shows that, compared to a baseline model and the dropout method, our method achieves superior performance over noisy inputs and out-of-domain data.