Guangming Xie

Ahsan Tanveer, Muhammad Hamza, Waqar Hussain Afridi et al.

Underwater manipulation often occurs under degraded visibility due to turbidity, glare, and gripper occlusion, limiting the reliability of vision-based perception during approach and grasping. In such settings, soft grippers are well suited for compliant interaction, but they typically lack an onboard pre-contact cue that can guide approach and closure when vision is unreliable. This extended abstract explores active electrosense as a lightweight sensing modality that can provide a proximity-like signal prior to contact by measuring perturbations of an applied electric field in conductive media. We instrument an octopus-inspired gripper with a discrete electrode layout and record multi-channel sensing voltages using off-the-shelf hardware. Simulation and tank experiments with a suspended conductive sphere show structured, object-dependent changes in the multi-electrode voltage readout relative to empty-water baselines, with detectability varying across excitation of 5 to 20 V and frequencies from 1 mHz to 1 kHz. These findings motivate systematic investigation of gripper-integrated electrosense as a complementary pre-contact cue for underwater soft manipulation.

Jianing Bai, Tianhao Zhang, Guangming Xie · pku

Congestion Control (CC), as the core networking task to efficiently utilize network capacity, received great attention and widely used in various Internet communication applications such as 5G, Internet-of-Things, UAN, and more. Various CC algorithms have been proposed both on network and transport layers such as Active Queue Management (AQM) algorithm and Transmission Control Protocol (TCP) congestion control mechanism. But it is hard to model dynamic AQM/TCP system and cooperate two algorithms to obtain excellent performance under different communication scenarios. In this paper, we explore the performance of multi-agent reinforcement learning-based cross-layer congestion control algorithms and present cooperation performance of two agents, known as MACC (Multi-agent Congestion Control). We implement MACC in NS3. The simulation results show that our scheme outperforms other congestion control combination in terms of throughput and delay, etc. Not only does it proves that networking protocols based on multi-agent deep reinforcement learning is efficient for communication managing, but also verifies that networking area can be used as new playground for machine learning algorithms.

Yueheng Li, Guangming Xie, Zongqing Lu

Due to practical constraints such as partial observability and limited communication, Centralized Training with Decentralized Execution (CTDE) has become the dominant paradigm in cooperative Multi-Agent Reinforcement Learning (MARL). However, existing CTDE methods often underutilize centralized training or lack theoretical guarantees. We propose Multi-Agent Guided Policy Optimization (MAGPO), a novel framework that better leverages centralized training by integrating centralized guidance with decentralized execution. MAGPO uses an auto-regressive joint policy for scalable, coordinated exploration and explicitly aligns it with decentralized policies to ensure deployability under partial observability. We provide theoretical guarantees of monotonic policy improvement and empirically evaluate MAGPO on 43 tasks across 6 diverse environments. Results show that MAGPO consistently outperforms strong CTDE baselines and matches or surpasses fully centralized approaches, offering a principled and practical solution for decentralized multi-agent learning. Our code and experimental data can be found in https://github.com/liyheng/MAGPO.

Yueheng Li, Guangming Xie, Zongqing Lu

Reinforcement Learning (RL) in partially observable environments poses significant challenges due to the complexity of learning under uncertainty. While additional information, such as that available in simulations, can enhance training, effectively leveraging it remains an open problem. To address this, we introduce Guided Policy Optimization (GPO), a framework that co-trains a guider and a learner. The guider takes advantage of privileged information while ensuring alignment with the learner's policy that is primarily trained via imitation learning. We theoretically demonstrate that this learning scheme achieves optimality comparable to direct RL, thereby overcoming key limitations inherent in existing approaches. Empirical evaluations show strong performance of GPO across various tasks, including continuous control with partial observability and noise, and memory-based challenges, significantly outperforming existing methods.

Tianhao Zhang, Yueheng Li, Shuai Li et al.

In this paper, the circle formation control problem is addressed for a group of cooperative underactuated fish-like robots involving unknown nonlinear dynamics and disturbances. Based on the reinforcement learning and cognitive consistency theory, we propose a decentralized controller without the knowledge of the dynamics of the fish-like robots. The proposed controller can be transferred from simulation to reality. It is only trained in our established simulation environment, and the trained controller can be deployed to real robots without any manual tuning. Simulation results confirm that the proposed model-free robust formation control method is scalable with respect to the group size of the robots and outperforms other representative RL algorithms. Several experiments in the real world verify the effectiveness of our RL-based approach for circle formation control.

Qinghao Wang, Ruijun Liu, Wei Wang et al.

Underwater communication is extremely challenging for small underwater robots which typically have stringent power and size constraints. In our previous work, we developed an artificial electrocommunication system which could be an alternative for the communication of small underwater robots. This paper further presents a new electrocommunication system that utilizes Binary Frequency Shift Keying (2FSK) modulation and deep-learning-based demodulation for underwater robots. We first derive an underwater electrocommunication model that covers both the near-field area and a large transition area outside of the near-field area. 2FSK modulation is adopted to improve the anti-interference ability of the electric signal. A deep learning algorithm is used to demodulate the electric signal by the receiver. Simulations and experiments show that with the same testing condition, the new communication system outperforms the previous system in both the communication distance and the data transmitting rate. In specific, the newly developed communication system achieves stable communication within the distance of 10 m at a data transfer rate of 5 Kbps with a power consumption of less than 0.1 W. The substantial increase in communication distance further improves the possibility of electrocommunication in underwater robotics.

Ci Chen, Yuanfang Wan, Baowei Li et al.

For heterogeneous unmanned systems composed of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs), using UAVs serve as eyes to assist UGVs in motion planning is a promising research direction due to the UAVs' vast view scope. However, due to UAVs flight altitude limitations, it may be impossible to observe the global map, and motion planning in the local map is a POMDP (Partially Observable Markov Decision Process) problem. This paper proposes a motion planning algorithm for heterogeneous unmanned system under partial observation from UAV without reconstruction of global maps, which consists of two parts designed for perception and decision-making, respectively. For the perception part, we propose the Grid Map Generation Network (GMGN), which is used to perceive scenes from UAV's perspective and classify the pathways and obstacles. For the decision-making part, we propose the Motion Command Generation Network (MCGN). Due to the addition of memory mechanism, MCGN has planning and reasoning abilities under partial observation from UAVs. We evaluate our proposed algorithm by comparing with baseline algorithms. The results show that our method effectively plans the motion of heterogeneous unmanned systems and achieves a relatively high success rate.

Xingwen Zheng, Ningzhe Hou, Pascal Johannes Daniel Dinjens et al.

Novel robotic grippers have captured increasing interests recently because of their abilities to adapt to varieties of circumstances and their powerful functionalities. Differing from traditional gripper with mechanical components-made fingers, novel robotic grippers are typically made of novel structures and materials, using a novel manufacturing process. In this paper, a novel robotic gripper with external frame and internal thermoplastic elastomer belt-made net is proposed. The gripper grasps objects using the friction between the net and objects. It has the ability of adaptive gripping through flexible contact surface. Stress simulation has been used to explore the regularity between the normal stress on the net and the deformation of the net. Experiments are conducted on a variety of objects to measure the force needed to reliably grip and hold the object. Test results show that the gripper can successfully grip objects with varying shape, dimensions, and textures. It is promising that the gripper can be used for grasping fragile objects in the industry or out in the field, and also grasping the marine organisms without hurting them.

Xingwen Zheng, Wei Wang, Liang Li et al.

The lateral line enables fish to efficiently sense the surrounding environment, thus assisting flow-related fish behaviours. Inspired by this phenomenon, varieties of artificial lateral line systems (ALLSs) have been developed and applied to underwater robots. This article focuses on using the pressure sensor arrays based on ALLS-measured hydrodynamic pressure variations (HPVs) for estimating the relative state between two adjacent robotic fish with leader-follower formation. The relative states include the relative oscillating frequency, amplitude, and offset of the upstream robotic fish to the downstream robotic fish, the relative vertical distance, the relative yaw angle, the relative pitch angle, and the relative roll angle between the two adjacent robotic fish. Regression model between the ALLS-measured and the mentioned relative states is investigated, and regression model-based relative state estimation is conducted. Specifically, two criteria are proposed firstly to investigate not only the sensitivity of each pressure sensor to the variations of relative state but also the insufficiency and redundancy of the pressure sensors. And thus the pressure sensors used for regression analysis are determined. Then four typical regression methods, including random forest algorithm, support vector regression, back propagation neural network, and multiple linear regression method are used for establishing regression models between the ALLS-measured HPVs and the relative states. Then regression effects of the four methods are compared and discussed. Finally, random forest-based method, which has the best regression effect, is used to estimate relative yaw angle and oscillating amplitude using the ALLS-measured HPVs and exhibits excellent estimation performance. This work contributes to local relative estimation for a group of underwater robots, which has always been a challenge.

Xingwen Zheng, Minglei Xiong, Junzheng Zheng et al.

Dynamic modeling has been capturing attention for its fundamentality in precise locomotion analyses and control of underwater robots. However, the existing researches have mainly focused on investigating two-dimensional motion of underwater robots, and little attention has been paid to three-dimensional dynamic modeling, which is just what we focus on. In this article, a three-dimensional dynamic model of an active-tail-actuated robotic fish with a barycentre regulating mechanism is built by combining Newton's second law for linear motion and Euler's equation for angular motion. The model parameters are determined by three-dimensional computer-aided design (CAD) software SolidWorks, HyperFlow-based computational fluid dynamics (CFD) simulation, and grey-box model estimation method. Both kinematic experiments with a prototype and numerical simulations are applied to validate the accuracy of the dynamic model mutually. Based on the dynamic model, multiple three-dimensional motions, including rectilinear motion, turning motion, gliding motion, and spiral motion, are analyzed. The experimental and simulation results demonstrate the effectiveness of the proposed model in evaluating the trajectory, attitude, and motion parameters, including the velocity, turning radius, angular velocity, etc., of the robotic fish.

Yufan Zhai, Xingwen Zheng, Guangming Xie

Any phenomenon in nature is potential to be an inspiration for us to propose new ideas. Lateral line is a typical example which has attracted more interest in recent years. With the aid of lateral line, fish is capable of acquiring fluid information around, which is of great significance for them to survive, communicate and hunt underwater. In this paper, we briefly introduce the morphology and mechanism of the lateral line first. Then we focus on the development of artificial lateral line which typically consists of an array of sensors and can be installed on underwater robots. A series of sensors inspired by the lateral line with different sensing principles have been summarized. And then the applications of artificial lateral line system in hydrodynamic environment sensing and vortices detection, dipole oscillation source detection, and autonomous control of underwater robots have been surveyed. In addition, the existing problems and future foci in the field have been further discussed in detail. The current works and future foci have demonstrated that artificial lateral line has great potentials of research and contributes to the applications of underwater robots.