Prism: Spectral Parameter Sharing for Multi-Agent Reinforcement Learning
This addresses scalability and diversity issues in MARL for applications like robotics and gaming, representing an incremental improvement over existing methods.
The paper tackled the problem of homogeneous behaviors in multi-agent reinforcement learning (MARL) due to conventional parameter sharing, proposing Prism, a framework using spectral domain representation via SVD to induce diversity while maintaining scalability, achieving competitive performance with superior resource efficiency on benchmarks like LBF, SMACv2, and MaMuJoCo.
Parameter sharing is a key strategy in multi-agent reinforcement learning (MARL) for improving scalability, yet conventional fully shared architectures often collapse into homogeneous behaviors. Recent methods introduce diversity through clustering, pruning, or masking, but typically compromise resource efficiency. We propose Prism, a parameter sharing framework that induces inter-agent diversity by representing shared networks in the spectral domain via singular value decomposition (SVD). All agents share the singular vector directions while learning distinct spectral masks on singular values. This mechanism encourages inter-agent diversity and preserves scalability. Extensive experiments on both homogeneous (LBF, SMACv2) and heterogeneous (MaMuJoCo) benchmarks show that Prism achieves competitive performance with superior resource efficiency.