Cutting Recursive Autoencoder Trees
This work addresses interpretability in deep learning for NLP, offering an incremental improvement by pruning model structures based on empirical tests.
The paper analyzes the Semi-Supervised Recursive Autoencoder to determine if its structure can be simplified without harming classification accuracy, finding that significant reduction is possible for certain tasks while maintaining performance.
Deep Learning models enjoy considerable success in Natural Language Processing. While deep architectures produce useful representations that lead to improvements in various tasks, they are often difficult to interpret. This makes the analysis of learned structures particularly difficult. In this paper, we rely on empirical tests to see whether a particular structure makes sense. We present an analysis of the Semi-Supervised Recursive Autoencoder, a well-known model that produces structural representations of text. We show that for certain tasks, the structure of the autoencoder can be significantly reduced without loss of classification accuracy and we evaluate the produced structures using human judgment.