Parameter-Efficient Neural Question Answering Models via Graph-Enriched Document Representations
This addresses the need for more efficient NLP models, particularly in question answering, by offering a parameter-efficient alternative to large language models, though it appears incremental as it builds on existing graph network methods.
The paper tackles the problem of high computational cost in NLP systems by proposing a graph convolutional document representation for question answering, achieving comparable or superior performance to state-of-the-art models while using less than 5% of trainable parameters.
As the computational footprint of modern NLP systems grows, it becomes increasingly important to arrive at more efficient models. We show that by employing graph convolutional document representation, we can arrive at a question answering system that performs comparably to, and in some cases exceeds the SOTA solutions, while using less than 5\% of their resources in terms of trainable parameters. As it currently stands, a major issue in applying GCNs to NLP is document representation. In this paper, we show that a GCN enriched document representation greatly improves the results seen in HotPotQA, even when using a trivial topology. Our model (gQA), performs admirably when compared to the current SOTA, and requires little to no preprocessing. In Shao et al. 2020, the authors suggest that graph networks are not necessary for good performance in multi-hop QA. In this paper, we suggest that large language models are not necessary for good performance by showing a naïve implementation of a GCN performs comparably to SoTA models based on pretrained language models.