Learning to Coordinate Multiple Reinforcement Learning Agents for Diverse Query Reformulation
This work addresses query reformulation in retrieval and QA tasks, offering an incremental improvement over existing methods.
The paper tackles the problem of learning diverse query reformulation strategies for document retrieval and question answering by proposing a multi-agent reinforcement learning framework with specialized sub-agents and a meta-agent. The result is faster learning due to high parallelizability and better generalization performance than strong baselines, attributed to increased diversity in strategies.
We propose a method to efficiently learn diverse strategies in reinforcement learning for query reformulation in the tasks of document retrieval and question answering. In the proposed framework an agent consists of multiple specialized sub-agents and a meta-agent that learns to aggregate the answers from sub-agents to produce a final answer. Sub-agents are trained on disjoint partitions of the training data, while the meta-agent is trained on the full training set. Our method makes learning faster, because it is highly parallelizable, and has better generalization performance than strong baselines, such as an ensemble of agents trained on the full data. We show that the improved performance is due to the increased diversity of reformulation strategies.