Fully Decentralized Policies for Multi-Agent Systems: An Information Theoretic Approach
This addresses coordination challenges in multi-agent systems for applications like robotics or autonomous networks, but it is incremental as it builds on existing optimization and information theory concepts.
The paper tackles the problem of learning cooperative policies for multi-agent systems without communication by developing an information theoretic framework based on rate distortion theory to analyze how well decentralized policies can reconstruct the optimal centralized solution, and it extends this to determine beneficial communication links.
Learning cooperative policies for multi-agent systems is often challenged by partial observability and a lack of coordination. In some settings, the structure of a problem allows a distributed solution with limited communication. Here, we consider a scenario where no communication is available, and instead we learn local policies for all agents that collectively mimic the solution to a centralized multi-agent static optimization problem. Our main contribution is an information theoretic framework based on rate distortion theory which facilitates analysis of how well the resulting fully decentralized policies are able to reconstruct the optimal solution. Moreover, this framework provides a natural extension that addresses which nodes an agent should communicate with to improve the performance of its individual policy.