Words or Characters? Fine-grained Gating for Reading Comprehension
This work addresses a specific bottleneck in reading comprehension models for NLP researchers, offering an incremental improvement over existing methods.
The paper tackled the problem of suboptimal combination of word-level and character-level representations in reading comprehension by introducing a fine-grained gating mechanism that dynamically combines these representations based on word properties, achieving new state-of-the-art results on the Children's Book Test dataset.
Previous work combines word-level and character-level representations using concatenation or scalar weighting, which is suboptimal for high-level tasks like reading comprehension. We present a fine-grained gating mechanism to dynamically combine word-level and character-level representations based on properties of the words. We also extend the idea of fine-grained gating to modeling the interaction between questions and paragraphs for reading comprehension. Experiments show that our approach can improve the performance on reading comprehension tasks, achieving new state-of-the-art results on the Children's Book Test dataset. To demonstrate the generality of our gating mechanism, we also show improved results on a social media tag prediction task.