Zhiyong Sun

Ruohan Wang, Zhiyong Sun, Sofie Haesaert

Formal safety guarantees on the synthesis of controllers for stochastic systems can be obtained using correct-by-design approaches. These approaches often use abstractions as finite-state Markov Decision Processes. As the state space of these MDPs grows, the curse of dimensionality makes the computational and memory cost of the probabilistic guarantees, quantified with dynamic programming, scale exponentially. In this work, we leverage decoupled dynamics and unravel, via dynamic programming operations, a tree structure in the Canonical Polyadic Decomposition (CPD) of the value functions. For discrete-time stochastic systems with syntactically co-safe linear temporal logic (scLTL) specifications, we provide provable probabilistic safety guarantees and significantly alleviate the computational burden. We provide an initial validation of the theoretical results on several typical case studies and showcase that the uncovered tree structure enables efficient reductions in the computational burden.

Toshiharu Sugie, Brian D. O. Anderson, Zhiyong Sun et al.

This paper considers a formation shape control problem for point agents in a two-dimensional ambient space, where the control is distributed, is based on achieving desired distances between nominated agent pairs, and avoids the possibility of reflection ambiguities. This has potential applications for large-scale multi-agent systems having simple information exchange structure. One solution to this type of problem, applicable to formations with just three or four agents, was recently given by considering a potential function which consists of both distance error and signed triangle area terms. However, it seems to be challenging to apply it to formations with more than four agents. This paper shows a hierarchical control strategy which can be applicable to any number of agents based on the above type of potential function and a formation shaping incorporating a grouping of equilateral triangles, so that all controlled distances are in fact the same. A key analytical result and some numerical results are shown to demonstrate the effectiveness of the proposed method.

Zhiyong Sun, Hector Garcia de Marina, Brian D. O. Anderson et al.

In this paper, we discuss quantization effects in rigid formation control systems when target formations are described by inter-agent distances. Because of practical sensing and measurement constraints, we consider in this paper distance measurements in their quantized forms. We show that under gradient-based formation control, in the case of uniform quantization, the distance errors converge locally to a bounded set whose size depends on the quantization error, while in the case of logarithmic quantization, all distance errors converge locally to zero. A special quantizer involving the signum function is then considered with which all agents can only measure coarse distances in terms of binary information. In this case, the formation converges locally to a target formation within a finite time. Lastly, we discuss the effect of asymmetric uniform quantization on rigid formation control.

Hengxu Zhang, Pengpeng Liang, Zhiyong Sun et al.

Both CNN-based and Transformer-based object detection with bounding box representation have been extensively studied in computer vision and medical image analysis, but circular object detection in medical images is still underexplored. Inspired by the recent anchor free CNN-based circular object detection method (CircleNet) for ball-shape glomeruli detection in renal pathology, in this paper, we present CircleFormer, a Transformer-based circular medical object detection with dynamic anchor circles. Specifically, queries with circle representation in Transformer decoder iteratively refine the circular object detection results, and a circle cross attention module is introduced to compute the similarity between circular queries and image features. A generalized circle IoU (gCIoU) is proposed to serve as a new regression loss of circular object detection as well. Moreover, our approach is easy to generalize to the segmentation task by adding a simple segmentation branch to CircleFormer. We evaluate our method in circular nuclei detection and segmentation on the public MoNuSeg dataset, and the experimental results show that our method achieves promising performance compared with the state-of-the-art approaches. The effectiveness of each component is validated via ablation studies as well. Our code is released at https://github.com/zhanghx-iim-ahu/CircleFormer.

Bomin Jiang, Zhiyong Sun, Brian D. O. Anderson et al.

Most current results on coverage control using mobile sensors require that one partitioned cell is associated with precisely one sensor. In this paper, we consider a class of coverage control problems involving higher order Voronoi partitions, motivated by applications where more than one sensor is required to monitor and cover one cell. Such applications are frequent in scenarios requiring the sensors to localize targets. We introduce a framework depending on a coverage performance function incorporating higher order Voronoi cells and then design a gradient-based controller which allows the multi-sensor system to achieve a local equilibrium in a distributed manner. The convergence properties are studied and related to Lloyd algorithm. We study also the extension to coverage of a discrete set of points. In addition, we provide a number of real world scenarios where our framework can be applied. Simulation results are also provided to show the controller performance.

Yongle Luo, Yuxin Wang, Kun Dong et al.

Exploration with sparse rewards remains a challenging research problem in reinforcement learning (RL). Especially for sequential object manipulation tasks, the RL agent always receives negative rewards until completing all sub-tasks, which results in low exploration efficiency. To solve these tasks efficiently, we propose a novel self-guided continual RL framework, RelayHER (RHER). RHER first decomposes a sequential task into new sub-tasks with increasing complexity and ensures that the simplest sub-task can be learned quickly by utilizing Hindsight Experience Replay (HER). Secondly, we design a multi-goal & multi-task network to learn these sub-tasks simultaneously. Finally, we propose a Self-Guided Exploration Strategy (SGES). With SGES, the learned sub-task policy will guide the agent to the states that are helpful to learn more complex sub-task with HER. By this self-guided exploration and relay policy learning, RHER can solve these sequential tasks efficiently stage by stage. The experimental results show that RHER significantly outperforms vanilla-HER in sample-efficiency on five singleobject and five complex multi-object manipulation tasks (e.g., Push, Insert, ObstaclePush, Stack, TStack, etc.). The proposed RHER has also been applied to learn a contact-rich push task on a physical robot from scratch, and the success rate reached 10/10 with only 250 episodes.

Zhiyong Sun, Toshiharu Sugie

Multi-agent coordination control usually involves a potential function that encodes information of a global control task, while the control input for individual agents is often designed by a gradient-based control law. The property of Hessian matrix associated with a potential function plays an important role in the stability analysis of equilibrium points in gradient-based coordination control systems. Therefore, the identification of Hessian matrix in gradient-based multi-agent coordination systems becomes a key step in multi-agent equilibrium analysis. However, very often the identification of Hessian matrix via the entry-wise calculation is a very tedious task and can easily introduce calculation errors. In this paper we present some general and fast approaches for the identification of Hessian matrix based on matrix differentials and calculus rules, which can easily derive a compact form of Hessian matrix for multi-agent coordination systems. We also present several examples on Hessian identification for certain typical potential functions involving edge-tension distance functions and triangular-area functions, and illustrate their applications in the context of distributed coordination and formation control.

Lin-Chi Wu, Zengjie Zhang, Sofie Haesaert et al.

Reinforcement learning (RL) is an effective approach to motion planning in autonomous driving, where an optimal driving policy can be automatically learned using the interaction data with the environment. Nevertheless, the reward function for an RL agent, which is significant to its performance, is challenging to be determined. The conventional work mainly focuses on rewarding safe driving states but does not incorporate the awareness of risky driving behaviors of the vehicles. In this paper, we investigate how to use risk-aware reward shaping to leverage the training and test performance of RL agents in autonomous driving. Based on the essential requirements that prescribe the safety specifications for general autonomous driving in practice, we propose additional reshaped reward terms that encourage exploration and penalize risky driving behaviors. A simulation study in OpenAI Gym indicates the advantage of risk-aware reward shaping for various RL agents. Also, we point out that proximal policy optimization (PPO) is likely to be the best RL method that works with risk-aware reward shaping.

Ruohan Wang, Siyuan Liu, Zhiyong Sun et al.

Discrete-time stochastic systems with continuous spaces are hard to verify and control, even with MDP abstractions due to the curse of dimensionality. We propose an abstraction-based framework with robust dynamic programming mappings that deliver control strategies with provable lower bounds on temporal-logic satisfaction, quantified via approximate stochastic simulation relations. Exploiting decoupled dynamics, we reveal a Canonical Polyadic Decomposition tensor structure in value functions that makes dynamic programming scalable. The proposed method provides correct-by-design probabilistic guarantees for temporal logic specifications. We validate our results on continuous-state linear stochastic systems.

Hector Garcia de Marina, Zhiyong Sun, Shaoshuai Mou

This paper investigates the stability of distance-based \textit{flexible} undirected formations in the plane. Without rigidity, there exists a set of connected shapes for given distance constraints, which is called the ambit. We show that a flexible formation can lose its flexibility, or equivalently may reduce the degrees of freedom of its ambit, if a small disturbance is introduced in the range sensor of the agents. The stability of the disturbed equilibrium can be characterized by analyzing the eigenvalues of the linearized augmented error system. Unlike infinitesimally rigid formations, the disturbed desired equilibrium can be turned unstable regardless of how small the disturbance is. We finally present two examples of how to exploit these disturbances as design parameters. The first example shows how to combine rigid and flexible formations such that some of the agents can move freely in the desired and locally stable ambit. The second example shows how to achieve a specific shape with fewer edges than the necessary for the standard controller in rigid formations.

Zhiyong Sun, Changbin, Yu

In this paper we study coupled dynamical systems and investigate dimension properties of the subspace spanned by solutions of each individual system. Relevant problems on \textit{collinear dynamical systems} and their variations are discussed recently by Montenbruck et. al. in \cite{collinear2017SCL}, while in this paper we aim to provide a unified analysis to derive the dimensional-invariance principles for networked coupled systems, and to generalize the invariance principles for networked systems with more general forms of coupling terms. To be specific, we consider two types of coupled systems, one with scalar couplings and the other with matrix couplings. Via the \textit{rank-preserving flow theory}, we show that any scalar-coupled dynamical system (with constant, time-varying or state-dependent couplings) possesses the dimensional-invariance principles, in that the dimension of the subspace spanned by the individual systems' solutions remains invariant. For coupled dynamical systems with matrix coefficients/couplings, necessary and sufficient conditions (for constant, time-varying and state-dependent couplings) are given to characterize dimensional-invariance principles. The proofs via a rank-preserving matrix flow theory in this paper simplify the analysis in \cite{collinear2017SCL}, and we also extend the invariance principles to the cases of time-varying couplings and state-dependent couplings. Furthermore, subspace-preserving property and signature-preserving flows are also developed for coupled networked systems with particular coupling terms. These invariance principles provide insightful characterizations to analyze transient behaviors and solution evolutions for a large family of coupled systems, such as multi-agent consensus dynamics, distributed coordination systems, formation control systems, among others.

Teun van de Laar, Zengjie Zhang, Shuhao Qi et al.

It has been an ambition of many to control a robot for a complex task using natural language (NL). The rise of large language models (LLMs) makes it closer to coming true. However, an LLM-powered system still suffers from the ambiguity inherent in an NL and the uncertainty brought up by LLMs. This paper proposes a novel LLM-based robot motion planner, named \textit{VernaCopter}, with signal temporal logic (STL) specifications serving as a bridge between NL commands and specific task objectives. The rigorous and abstract nature of formal specifications allows the planner to generate high-quality and highly consistent paths to guide the motion control of a robot. Compared to a conventional NL-prompting-based planner, the proposed VernaCopter planner is more stable and reliable due to less ambiguous uncertainty. Its efficacy and advantage have been validated by two small but challenging experimental scenarios, implying its potential in designing NL-driven robots.

Hector Garcia de Marina, Zhiyong Sun, Murat Bronz et al.

In this paper we propose an algorithm for stabilizing circular formations of fixed-wing UAVs with constant speeds. The algorithm is based on the idea of tracking circles with different radii in order to control the inter-vehicle phases with respect to a target circumference. We prove that the desired equilibrium is exponentially stable and thanks to the guidance vector field that guides the vehicles, the algorithm can be extended to other closed trajectories. One of the main advantages of this approach is that the algorithm guarantees the confinement of the team in a specific area, even when communications or sensing among vehicles are lost. We show the effectiveness of the algorithm with an actual formation flight of three aircraft. The algorithm is ready to use for the general public in the open-source Paparazzi autopilot.

Ming Li, Zhiyong Sun, Siep Weiland

This paper revisits a classical challenge in the design of stabilizing controllers for nonlinear systems with a norm-bounded input constraint. By extending Lin-Sontag's universal formula and introducing a generic (state-dependent) scaling term, a unifying controller design method is proposed. The incorporation of this generic scaling term gives a unified controller and enables the derivation of alternative universal formulas with various favorable properties, which makes it suitable for tailored control designs to meet specific requirements and provides versatility across different control scenarios. Additionally, we present a constructive approach to determine the optimal scaling term, leading to an explicit solution to an optimization problem, named optimization-based universal formula. The resulting controller ensures asymptotic stability, satisfies a norm-bounded input constraint, and optimizes a predefined cost function. Finally, the essential properties of the unified controllers are analyzed, including smoothness, continuity at the origin, stability margin, and inverse optimality. Simulations validate the approach, showcasing its effectiveness in addressing a challenging stabilizing control problem of a nonlinear system.

Ming Li, Zhiyong Sun

In this paper, we demonstrate that controllers designed by artificial potential fields (APFs) can be derived from reciprocal control barrier function quadratic program (RCBF-QP) safety filters. By integrating APFs within the RCBF-QP framework, we explicitly establish the relationship between these two approaches. Specifically, we first introduce the concepts of tightened control Lyapunov functions (T-CLFs) and tightened reciprocal control barrier functions (T-RCBFs), each of which incorporates a flexible auxiliary function. We then utilize an attractive potential field as a T-CLF to guide the nominal controller design, and a repulsive potential field as a T-RCBF to formulate an RCBF-QP safety filter. With appropriately chosen auxiliary functions, we show that controllers designed by APFs and those derived by RCBF-QP safety filters are equivalent. Based on this insight, we further generalize the APF-based controllers (equivalently, RCBF-QP safety filter-based controllers) to more general scenarios without restricting the choice of auxiliary functions. Finally, we present a collision avoidance example to clearly illustrate the connection and equivalence between the two methods.

Zhiyong Sun, Marcus Greiff, Anders Robertsson et al.

We consider the problem of cooperative motion coordination for multiple heterogeneous mobile vehicles subject to various constraints. These include nonholonomic motion constraints, constant speed constraints, holonomic coordination constraints, and equality/inequality geometric constraints. We develop a general framework involving differential-algebraic equations and viability theory to determine coordination feasibility for a coordinated motion control under heterogeneous vehicle dynamics and different types of coordination task constraints. If a coordinated motion solution exists for the derived differential-algebraic equations and/or inequalities, a constructive algorithm is proposed to derive an equivalent dynamical system that generates a set of feasible coordinated motions for each individual vehicle. In case studies on coordinating two vehicles, we derive analytical solutions to motion generation for two-vehicle groups consisting of car-like vehicles, unicycle vehicles, or vehicles with constant speeds, which serve as benchmark coordination tasks for more complex vehicle groups. The motion generation algorithm is well-backed by simulation data for a wide variety of coordination situations involving heterogeneous vehicles. We then extend the vehicle control framework to deal with the cooperative coordination problem with time-varying coordination tasks and leader-follower structure. We show several simulation experiments on multi-vehicle coordination under various constraints to validate the theory and the effectiveness of the proposed schemes.

Weijia Yao, Hector Garcia de Marina, Zhiyong Sun et al.

It is essential in many applications to impose a scalable coordinated motion control on a large group of mobile robots, which is efficient in tasks requiring repetitive execution, such as environmental monitoring. In this paper, we design a guiding vector field to guide multiple robots to follow possibly different desired paths while coordinating their motions. The vector field uses a path parameter as a virtual coordinate that is communicated among neighboring robots. Then, the virtual coordinate is utilized to control the relative parametric displacement between robots along the paths. This enables us to design a saturated control algorithm for a Dubins-car-like model. The algorithm is distributed, scalable, and applicable for any smooth paths in an $n$-dimensional configuration space, and global convergence is guaranteed. Simulations with up to fifty robots and outdoor experiments with fixed-wing aircraft validate the theoretical results.

Yongle Luo, Kun Dong, Lili Zhao et al.

Efficient and effective learning is one of the ultimate goals of the deep reinforcement learning (DRL), although the compromise has been made in most of the time, especially for the application of robot manipulations. Learning is always expensive for robot manipulation tasks and the learning effectiveness could be affected by the system uncertainty. In order to solve above challenges, in this study, we proposed a simple but powerful reward shaping method, namely Dense2Sparse. It combines the advantage of fast convergence of dense reward and the noise isolation of the sparse reward, to achieve a balance between learning efficiency and effectiveness, which makes it suitable for robot manipulation tasks. We evaluated our Dense2Sparse method with a series of ablation experiments using the state representation model with system uncertainty. The experiment results show that the Dense2Sparse method obtained higher expected reward compared with the ones using standalone dense reward or sparse reward, and it also has a superior tolerance of system uncertainty.

Xiuhui Peng, Zhiyong Sun, Kexin Guo et al.

This paper addresses forward motion control for trajectory tracking and mobile formation coordination for a group of non-holonomic vehicles on SE(2). Firstly, by constructing an intermediate attitude variable which involves vehicles' position information and desired attitude, the translational and rotational control inputs are designed in two stages to solve the trajectory tracking problem. Secondly, the coordination relationships of relative positions and headings are explored thoroughly for a group of non-holonomic vehicles to maintain a mobile formation with rigid body motion constraints. We prove that, except for the cases of parallel formation and translational straight line formation, a mobile formation with strict rigid-body motion can be achieved if and only if the ratios of linear speed to angular speed for each individual vehicle are constants. Motion properties for mobile formation with weak rigid-body motion are also demonstrated. Thereafter, based on the proposed trajectory tracking approach, a distributed mobile formation control law is designed under a directed tree graph. The performance of the proposed controllers is validated by both numerical simulations and experiments.

Zhiyong Sun, Qingchen Liu, Na Huang et al.

This paper discusses cooperative stabilization control of rigid formations via an event-based approach. We first design a centralized event-based formation control system, in which a central event controller determines the next triggering time and broadcasts the event signal to all the agents for control input update. We then build on this approach to propose a distributed event control strategy, in which each agent can use its local event trigger and local information to update the control input at its own event time. For both cases, the triggering condition, event function and triggering behavior are discussed in detail, and the exponential convergence of the event-based formation system is guaranteed.

Marcus Greiff, Zhiyong Sun, Anders Robertsson et al.

Controlled invariant set and viability regulation of dynamical control systems have played important roles in many control and coordination applications. In this paper we develop a temporal viability regulation theory for general dynamical control systems, and in particular for control affine systems. The time-varying viable set is parameterized by time-varying constraint functions, with the aim to regulate a dynamical control system to be invariant in the time-varying viable set so that temporal state-dependent constraints are enforced. We consider both time-varying equality and inequality constraints in defining a temporal viable set. We also present sufficient conditions for the existence of feasible control input for the control affine systems. The developed temporal viability regulation theory is applied to mobile vehicle coordination.

Zhiyong Sun, Hector Garcia de Marina, Brian D. O. Anderson et al.

This paper considers a collaborative tracking control problem using a group of fixed-wing unmanned aerial vehicles (UAVs) with constant and non-identical speeds. The dynamics of fixed-wing UAVs are modelled by unicycle-type equations with nonholonomic constraints, assuming that UAVs fly at constant altitudes in the nominal operation mode. The controller is designed such that all fixed-wing UAVs as a group can collaboratively track a desired target's position and velocity. We first present conditions on the relative speeds of tracking UAVs and the target to ensure that the tracking objective can be achieved when UAVs are subject to constant speed constraints. We construct a reference velocity that includes both the target's velocity and position as feedback, which is to be tracked by the group centroid. In this way, all vehicles' headings are controlled such that the group centroid follows a reference trajectory that successfully tracks the target's trajectory. A spacing controller is further devised to ensure that all vehicles stay close to the group centroid trajectory. Trade-offs in the controller design and performance limitations of the target tracking control due to the constant-speed constraint are also discussed in detail. Experimental results with three fixed-wing UAVs tracking a target rotorcraft are provided.

Zhiyong Sun, Marcus Greiff, Anders Robertsson et al.

We consider the problem of feasible coordination control for multiple homogeneous or heterogeneous mobile vehicles subject to various constraints (nonholonomic motion constraints, holonomic coordination constraints, equality/inequality constraints etc). We develop a general framework involving differential-algebraic equations and viability theory to describe and determine coordination feasibility for a coordinated motion control under heterogeneous vehicle dynamics and different types of coordination constraints. If a solution exists for the derived differential-algebraic equations and/or inequalities, a heuristic algorithm is proposed for generating feasible trajectories for each individual vehicle. In case studies on coordinating two vehicles, we derive analytical solutions to motion generation for two-vehicle groups consisting of car-like vehicles, unicycle vehicles, or vehicles with constant speeds, which serve as benchmark coordination tasks for more complex vehicle groups. We show several simulation experiments on multi-vehicle coordination under various constraints to validate the theory and the effectiveness of the proposed schemes.

Shiyu Zhao, Zhiyong Sun, Daniel Zelazo et al.

This paper addresses the problem of constructing bearing rigid networks in arbitrary dimensions. We first show that the bearing rigidity of a network is a generic property that is critically determined by the underlying graph of the network. A new notion termed generic bearing rigidity is defined for graphs. If the underlying graph of a network is generically bearing rigid, then the network is bearing rigid for almost all configurations; otherwise, the network is not bearing rigid for any configuration. As a result, the key to construct bearing rigid networks is to construct generically bearing rigid graphs. The main contribution of this paper is to prove that Laman graphs, which can be generated by the Henneberg construction, are generically bearing rigid in arbitrary dimensions. As a consequence, if the underlying graph of a network is Laman, the network is bearing rigid for almost all configurations in arbitrary dimensions.

Hector Garcia de Marina, Zhiyong Sun, Ming Cao et al.

In formation control, triangular formations consisting of three autonomous agents serve as a class of benchmarks that can be used to test and compare the performances of different controllers. We present an algorithm that combines the advantages of both position- and distance-based gradient descent control laws. For example, only two pairs of neighboring agents need to be controlled, agents can work in their own local frame of coordinates and the orientation of the formation with respect to a global frame of coordinates is not prescribed. We first present a novel technique based on adding artificial biases to neighboring agents' range sensors such that their eventual positions correspond to a collinear configuration. Right after, a small modification in the bias terms by introducing a prescribed rotation matrix will allow the control of the bearing of the neighboring agents.