Learning to Reason With Adaptive Computation
This work addresses the challenge of efficient and interpretable reasoning in NLP tasks, but it is incremental as it builds on existing adaptive computation methods with modest gains.
The paper tackled the problem of multi-hop inference in tasks like Recognising Textual Entailment and Machine Reading by using adaptive computation to learn the number of inference steps needed for examples of varying complexity, resulting in a small performance improvement over a non-adaptive model and providing insights into the model's reasoning process.
Multi-hop inference is necessary for machine learning systems to successfully solve tasks such as Recognising Textual Entailment and Machine Reading. In this work, we demonstrate the effectiveness of adaptive computation for learning the number of inference steps required for examples of different complexity and that learning the correct number of inference steps is difficult. We introduce the first model involving Adaptive Computation Time which provides a small performance benefit on top of a similar model without an adaptive component as well as enabling considerable insight into the reasoning process of the model.