When Do Decompositions Help for Machine Reading?
This work addresses the understudied problem of when decompositions are beneficial for machine reading, providing insights for researchers and practitioners, though it is incremental as it unifies and analyzes existing work.
The study investigated when decompositions help in machine reading, finding they improve exact match scores by several points in few-shot scenarios but become unhelpful or detrimental with a few hundred or more examples, suggesting models can learn decompositions implicitly with limited data.
Answering complex questions often requires multi-step reasoning in order to obtain the final answer. Most research into decompositions of complex questions involves open-domain systems, which have shown success in using these decompositions for improved retrieval. In the machine reading setting, however, work to understand when decompositions are helpful is understudied. We conduct experiments on decompositions in machine reading to unify recent work in this space, using a range of models and datasets. We find that decompositions can be helpful in the few-shot case, giving several points of improvement in exact match scores. However, we also show that when models are given access to datasets with around a few hundred or more examples, decompositions are not helpful (and can actually be detrimental). Thus, our analysis implies that models can learn decompositions implicitly even with limited data.