Developing cooperative policies for multi-stage tasks
This addresses the challenge of multi-stage task efficiency in reinforcement learning, though it appears incremental as it modifies existing methods for cooperation.
The paper tackled the problem of enabling consecutive reinforcement learning agents to cooperatively solve long time horizon multi-stage tasks, resulting in CSAC achieving at least 20% higher success rates than uncooperative policies and converging at least 4 times faster than a single agent.
This paper proposes the Cooperative Soft Actor Critic (CSAC) method of enabling consecutive reinforcement learning agents to cooperatively solve a long time horizon multi-stage task. This method is achieved by modifying the policy of each agent to maximise both the current and next agent's critic. Cooperatively maximising each agent's critic allows each agent to take actions that are beneficial for its task as well as subsequent tasks. Using this method in a multi-room maze domain, the cooperative policies were able to outperform both uncooperative policies as well as a single agent trained across the entire domain. CSAC achieved a success rate of at least 20\% higher than the uncooperative policies, and converged on a solution at least 4 times faster than the single agent.