Modular Approach to Machine Reading Comprehension: Mixture of Task-Aware Experts
This addresses the challenge of over-fitting and common-sense integration in machine reading comprehension, particularly for small datasets, though it appears incremental as it builds on multitask and transfer learning advancements.
The paper tackles the problem of common-sense learning in machine reading comprehension by proposing a Mixture of Task-Aware Experts Network, which trains different expert networks to capture various relationships in passage-question-choice triplets, achieving state-of-the-art results and reducing over-fitting on a relatively small dataset.
In this work we present a Mixture of Task-Aware Experts Network for Machine Reading Comprehension on a relatively small dataset. We particularly focus on the issue of common-sense learning, enforcing the common ground knowledge by specifically training different expert networks to capture different kinds of relationships between each passage, question and choice triplet. Moreover, we take inspi ration on the recent advancements of multitask and transfer learning by training each network a relevant focused task. By making the mixture-of-networks aware of a specific goal by enforcing a task and a relationship, we achieve state-of-the-art results and reduce over-fitting.