Xun Wang

Zhuoran Li, Xun Wang, Hai Zhong et al.

Generative models, especially diffusion and flow-based models, have been promising in offline multi-agent reinforcement learning. However, integrating powerful generative models into this framework poses unique challenges. In particular, diffusion and flow-based policies suffer from low sampling efficiency due to their iterative generation processes, making them impractical in time-sensitive or resource-constrained settings. To tackle these difficulties, we propose OM2P (Offline Multi-Agent Mean-Flow Policy), a novel offline MARL algorithm to achieve efficient one-step action sampling. To address the misalignment between generative objectives and reward maximization, we introduce a reward-aware optimization scheme that integrates a carefully-designed mean-flow matching loss with Q-function supervision. Additionally, we design a generalized timestep distribution and a derivative-free estimation strategy to reduce memory overhead and improve training stability. Empirical evaluations on Multi-Agent Particle and MuJoCo benchmarks demonstrate that OM2P achieves superior performance, with up to a 3.8x reduction in GPU memory usage and up to a 10.8x speed-up in training time. Our approach represents the first to successfully integrate mean-flow model into offline MARL, paving the way for practical and scalable generative policies in cooperative multi-agent settings.

Xun Wang, Mary-Anne Williams

Python based Robot Interactive Development Environment (PyRIDE) is a software that supports rapid \textit{interactive} programming of robot skills and behaviours on PR2/ROS (Robot Operating System) platform. One of the key features of PyRIDE is its interactive remotely accessible Python console that allows its users to program robots \textit{online} and in \textit{realtime} in the same way as using the standard Python interactive interpreter. It allows programs to be modified while they are running. PyRIDE is also a software integration framework that abstracts and aggregates disparate low level ROS software modules, e.g. arm joint motor controllers, and exposes their functionalities through a unified Python programming interface. PR2 programmers are able to experiment and develop robot behaviours without dealing with specific details of accessing underlying softwares and hardwares. PyRIDE provides a client-server mechanism that allows remote user access of the robot functionalities, e.g. remote robot monitoring and control, access real-time robot camera image data etc. This enables multi-modal human robot interactions using different devices and user interfaces. All these features are seamlessly integrated into one lightweight and portable middleware package. In this paper, we use four real life scenarios to demonstrate PyRIDE key features and illustrate the usefulness of software.