Self-Adaptive Hierarchical Sentence Model
This work addresses a fundamental challenge in NLP for tasks like classification by improving recursive models, though it appears incremental as it builds on existing hierarchical approaches.
The paper tackled the problem of modeling sentences at varying hierarchical stages in natural language processing by proposing AdaSent, a self-adaptive hierarchical sentence model that uses recursive gated composition and a competitive mechanism to mitigate gradient vanishing, resulting in superior classification performance on 5 benchmark datasets.
The ability to accurately model a sentence at varying stages (e.g., word-phrase-sentence) plays a central role in natural language processing. As an effort towards this goal we propose a self-adaptive hierarchical sentence model (AdaSent). AdaSent effectively forms a hierarchy of representations from words to phrases and then to sentences through recursive gated local composition of adjacent segments. We design a competitive mechanism (through gating networks) to allow the representations of the same sentence to be engaged in a particular learning task (e.g., classification), therefore effectively mitigating the gradient vanishing problem persistent in other recursive models. Both qualitative and quantitative analysis shows that AdaSent can automatically form and select the representations suitable for the task at hand during training, yielding superior classification performance over competitor models on 5 benchmark data sets.