The Emergence of Chunking Structures with Hierarchical RNN
This work addresses the problem of reducing reliance on manual annotations for syntactic tasks in NLP, representing an incremental advancement in unsupervised syntactic structure discovery.
The paper tackles unsupervised chunking in NLP by introducing a Hierarchical Recurrent Neural Network (HRNN) with a two-stage training process, resulting in notable improvements in performance on multiple datasets.
In Natural Language Processing (NLP), predicting linguistic structures, such as parsing and chunking, has mostly relied on manual annotations of syntactic structures. This paper introduces an unsupervised approach to chunking, a syntactic task that involves grouping words in a non-hierarchical manner. We present a Hierarchical Recurrent Neural Network (HRNN) designed to model word-to-chunk and chunk-to-sentence compositions. Our approach involves a two-stage training process: pretraining with an unsupervised parser and finetuning on downstream NLP tasks. Experiments on multiple datasets reveal a notable improvement of unsupervised chunking performance in both pretraining and finetuning stages. Interestingly, we observe that the emergence of the chunking structure is transient during the neural model's downstream-task training. This study contributes to the advancement of unsupervised syntactic structure discovery and opens avenues for further research in linguistic theory.