Compositional Sentence Representation from Character within Large Context Text
This work addresses sentence representation challenges for natural language processing tasks, particularly dialogue act classification, but is incremental as it builds on hierarchical and recurrent methods.
The paper tackles the problems of data sparsity in word embeddings and lack of inter-sentence dependency in sentence representation by proposing a Hierarchical Composition Recurrent Network (HCRN) that builds word representations from characters and embeds inter-sentence dependencies, achieving state-of-the-art performance with a test error rate of 22.7% on dialogue act classification.
This paper describes a Hierarchical Composition Recurrent Network (HCRN) consisting of a 3-level hierarchy of compositional models: character, word and sentence. This model is designed to overcome two problems of representing a sentence on the basis of a constituent word sequence. The first is a data-sparsity problem in word embedding, and the other is a no usage of inter-sentence dependency. In the HCRN, word representations are built from characters, thus resolving the data-sparsity problem, and inter-sentence dependency is embedded into sentence representation at the level of sentence composition. We adopt a hierarchy-wise learning scheme in order to alleviate the optimization difficulties of learning deep hierarchical recurrent network in end-to-end fashion. The HCRN was quantitatively and qualitatively evaluated on a dialogue act classification task. Especially, sentence representations with an inter-sentence dependency are able to capture both implicit and explicit semantics of sentence, significantly improving performance. In the end, the HCRN achieved state-of-the-art performance with a test error rate of 22.7% for dialogue act classification on the SWBD-DAMSL database.