Xiaoqiang Ren

Xiaoqiang Ren, Junfeng Wu, Karl H. Johansson et al.

Jointly optimal transmission power control and remote estimation over an infinite horizon is studied. A sensor observes a dynamic process and sends its observations to a remote estimator over a wireless fading channel characterized by a time-homogeneous Markov chain. The successful transmission probability depends on both the channel gains and the transmission power used by the sensor. The transmission power control rule and the remote estimator should be jointly designed, aiming to minimize an infinite-horizon cost consisting of the power usage and the remote estimation error. A first question one may ask is: Does this joint optimization problem have a solution? We formulate the joint optimization problem as an average cost belief-state Markov decision process and answer the question by proving that there exists an optimal deterministic and stationary policy. We then show that when the monitored dynamic process is scalar, the optimal remote estimates depend only on the most recently received sensor observation, and the optimal transmission power is symmetric and monotonically increasing with respect to the innovation error.

Shuang Wu, Xiaoqiang Ren, Qing-Shan Jia et al.

We consider optimal sensor scheduling with unknown communication channel statistics. We formulate two types of scheduling problems with the communication rate being a soft or hard constraint, respectively. We first present some structural results on the optimal scheduling policy using dynamic programming and assuming the channel statistics is known. We prove that the Q-factor is monotonic and submodular, which leads to the threshold-like structures in both types of problems. Then we develop a stochastic approximation and parameter learning frameworks to deal with the two scheduling problems with unknown channel statistics. We utilize their structures to design specialized learning algorithms. We prove the convergence of these algorithms. Performance improvement compared with the standard Q-learning algorithm is shown through numerical examples.

Xiaoqiang Ren, Yilin Mo, Jie Chen et al.

This paper studies static state estimation in multi-sensor settings, with a caveat that an unknown subset of the sensors are compromised by an adversary, whose measurements can be manipulated arbitrarily. The attacker is able to compromise $q$ out of $m$ sensors. A new performance metric, which quantifies the asymptotic decay rate for the probability of having an estimation error larger than $δ$, is proposed. We develop an optimal estimator for the new performance metric with a fixed $δ$, which is the Chebyshev center of a union of ellipsoids. We further provide an estimator that is optimal for every $δ$, for the special case where the sensors are homogeneous. Numerical examples are given to elaborate the results.

Xingkang He, Xiaoqiang Ren, Henrik Sandberg et al.

In this paper, we consider a secure distributed filtering problem for linear time-invariant systems with bounded noises and unstable dynamics under compromised observations. A malicious attacker is able to compromise a subset of the agents and manipulate the observations arbitrarily. We first propose a recursive distributed filter consisting of two parts at each time. The first part employs a saturation-like scheme, which gives a small gain if the innovation is too large. The second part is a consensus operation of state estimates among neighboring agents. A sufficient condition is then established for the boundedness of estimation error, which is with respect to network topology, system structure, and the maximal compromised agent subset. We further provide an equivalent statement, which connects to 2s-sparse observability in the centralized framework in certain scenarios, such that the sufficient condition is feasible. Numerical simulations are finally provided to illustrate the developed results.

Shuang Wu, Xiaoqiang Ren, Subhrakanti Dey et al.

This paper considers the problem of sensory data scheduling of multiple processes. There are $n$ independent linear time-invariant processes and a remote estimator monitoring all the processes. Each process is measured by a sensor, which sends its local state estimate to the remote estimator. The sizes of the packets are different due to different dimensions of each process, and thus it may take different lengths of time steps for the sensors to send their data. Because of bandwidth limitation, only a portion of all the sensors are allowed to transmit. Our goal is to minimize the average of estimation error covariance of the whole system at the remote estimator. The problem is formulated as a Markov decision process (MDP) with average cost over an infinite time horizon. We prove the existence of a deterministic and stationary policy for the problem. We also find that the optimal policy has a consistent behavior and threshold type structure. A numerical example is provided to illustrate our main results.

Shuang Wu, Xiaoqiang Ren, Yiguang Hong et al.

We consider the design of a fair sensor schedule for a number of sensors monitoring different linear time-invariant processes. The largest average remote estimation error among all processes is to be minimized. We first consider a general setup for the max-min fair allocation problem. By reformulating the problem as its equivalent form, we transform the fair resource allocation problem into a zero-sum game between a "judge" and a resource allocator. We propose an equilibrium seeking procedure and show that there exists a unique Nash equilibrium in pure strategy for this game. We then apply the result to the sensor scheduling problem and show that the max-min fair sensor scheduling policy can be achieved.

Heng Zhang, Guoxiang Zhao, Xiaoqiang Ren

Pursuit-evasion (PE) problem is a critical challenge in multi-robot systems (MRS). While reinforcement learning (RL) has shown its promise in addressing PE tasks, research has primarily focused on single-target pursuit, with limited exploration of multi-target encirclement, particularly in large-scale settings. This paper proposes a Transformer-Enhanced Reinforcement Learning (TERL) framework for large-scale multi-target encirclement. By integrating a transformer-based policy network with target selection, TERL enables robots to adaptively prioritize targets and safely coordinate robots. Results show that TERL outperforms existing RL-based methods in terms of encirclement success rate and task completion time, while maintaining good performance in large-scale scenarios. Notably, TERL, trained on small-scale scenarios (15 pursuers, 4 targets), generalizes effectively to large-scale settings (80 pursuers, 20 targets) without retraining, achieving a 100% success rate. The code and demonstration video are available at https://github.com/ApricityZ/TERL.

Jiajun Qian, Liang Xu, Xiaoqiang Ren et al.

Deep neural networks (DNN) are increasingly being used to learn controllers due to their excellent approximation capabilities. However, their black-box nature poses significant challenges to closed-loop stability guarantees and performance analysis. In this paper, we introduce a structured DNN-based controller for the trajectory tracking control of Lagrangian systems using backing techniques. By properly designing neural network structures, the proposed controller can ensure closed-loop stability for any compatible neural network parameters. In addition, improved control performance can be achieved by further optimizing neural network parameters. Besides, we provide explicit upper bounds on tracking errors in terms of controller parameters, which allows us to achieve the desired tracking performance by properly selecting the controller parameters. Furthermore, when system models are unknown, we propose an improved Lagrangian neural network (LNN) structure to learn the system dynamics and design the controller. We show that in the presence of model approximation errors and external disturbances, the closed-loop stability and tracking control performance can still be guaranteed. The effectiveness of the proposed approach is demonstrated through simulations.

Zili Zhang, Ziting Wen, Yiheng Qiang et al.

Single-photon LiDAR achieves high-precision 3D imaging in extreme environments through quantum-level photon detection technology. Current research primarily focuses on reconstructing 3D scenes from sparse photon events, whereas the semantic interpretation of single-photon images remains underexplored, due to high annotation costs and inefficient labeling strategies. This paper presents the first active learning framework for single-photon image classification. The core contribution is an imaging condition-aware sampling strategy that integrates synthetic augmentation to model variability across imaging conditions. By identifying samples where the model is both uncertain and sensitive to these conditions, the proposed method selectively annotates only the most informative examples. Experiments on both synthetic and real-world datasets show that our approach outperforms all baselines and achieves high classification accuracy with significantly fewer labeled samples. Specifically, our approach achieves 97% accuracy on synthetic single-photon data using only 1.5% labeled samples. On real-world data, we maintain 90.63% accuracy with just 8% labeled samples, which is 4.51% higher than the best-performing baseline. This illustrates that active learning enables the same level of classification performance on single-photon images as on classical images, opening doors to large-scale integration of single-photon data in real-world applications.

Shuaibing Lin, JiaLiang Qu, Zishuo Li et al.

Complex autonomous driving, such as drifting, requires high-precision and high-frequency pose information to ensure accuracy and safety, which is notably difficult when using only onboard sensors. In this paper, we propose a drift controller with two feedback control loops: sideslip controller that stabilizes the sideslip angle by tuning the front wheel steering angle, and circle controller that maintains a stable trajectory radius and circle center by controlling the wheel rotational speed. We use an extended Kalman filter to estimate the state. A robustified KASA algorithm is further proposed to accurately estimate the parameters of the circle (i.e., the center and radius) that best fits into the current trajectory. On the premise of the uniform circular motion of the vehicle in the process of stable drift, we use angle information instead of acceleration to describe the dynamic of the vehicle. We implement our method on a 1/10 scale race car. The car drifts stably with a given center and radius, which illustrates the effectiveness of our method.

Dafa Ren, Xiaoqiang Ren, Xiaofan Wang et al.

Grasping objects in cluttered scenarios is a challenging task in robotics. Performing pre-grasp actions such as pushing and shifting to scatter objects is a way to reduce clutter. Based on deep reinforcement learning, we propose a Fast-Learning Grasping (FLG) framework, that can integrate pre-grasping actions along with grasping to pick up objects from cluttered scenarios with reduced real-world training time. We associate rewards for performing moving actions with the change of environmental clutter and utilize a hybrid triggering method, leading to data-efficient learning and synergy. Then we use the output of an extended fully convolutional network as the value function of each pixel point of the workspace and establish an accurate estimation of the grasp probability for each action. We also introduce a mask function as prior knowledge to enable the agents to focus on the accurate pose adjustment to improve the effectiveness of collecting training data and, hence, to learn efficiently. We carry out pre-training of the FLG over simulated environment, and then the learnt model is transferred to the real world with minimal fine-tuning for further learning during actions. Experimental results demonstrate a 94% grasp success rate and the ability to generalize to novel objects. Compared to state-of-the-art approaches in the literature, the proposed FLG framework can achieve similar or higher grasp success rate with lesser amount of training in the real world. Supplementary video is available at https://youtu.be/e04uDLsxfDg.

Xiaoqiang Ren, Junfeng Wu, Subhrakanti Dey et al.

This paper considers optimal attack attention allocation on remote state estimation in multi-systems. Suppose there are $\mathtt{M}$ independent systems, each of which has a remote sensor monitoring the system and sending its local estimates to a fusion center over a packet-dropping channel. An attacker may generate noises to exacerbate the communication channels between sensors and the fusion center. Due to capacity limitation, at each time the attacker can exacerbate at most $\mathtt{N}$ of the $\mathtt{M}$ channels. The goal of the attacker side is to seek an optimal policy maximizing the estimation error at the fusion center. The problem is formulated as a Markov decision process (MDP) problem, and the existence of an optimal deterministic and stationary policy is proved. We further show that the optimal policy has a threshold structure, by which the computational complexity is reduced significantly. Based on the threshold structure, a myopic policy is proposed for homogeneous models and its optimality is established. To overcome the curse of dimensionality of MDP algorithms for general heterogeneous models, we further provide an asymptotically (as $\mathtt{M}$ and $\mathtt{N}$ go to infinity) optimal solution, which is easy to compute and implement. Numerical examples are given to illustrate the main results.

Xiaoqiang Ren, Karl H. Johansson, Dawei Shi et al.

The problem of quickest change detection with communication rate constraints is studied. A network of wireless sensors with limited computation capability monitors the environment and sends observations to a fusion center via wireless channels. At an unknown time instant, the distributions of observations at all the sensor nodes change simultaneously. Due to limited energy, the sensors cannot transmit at all the time instants. The objective is to detect the change at the fusion center as quickly as possible, subject to constraints on false detection and average communication rate between the sensors and the fusion center. A minimax formulation is proposed. The cumulative sum (CuSum) algorithm is used at the fusion center and censoring strategies are used at the sensor nodes. The censoring strategies, which are adaptive to the CuSum statistic, are fed back by the fusion center. The sensors only send observations that fall into prescribed sets to the fusion center. This CuSum adaptive censoring (CuSum-AC) algorithm is proved to be an equalizer rule and to be globally asymptotically optimal for any positive communication rate constraint, as the average run length to false alarm goes to infinity. It is also shown, by numerical examples, that the CuSum-AC algorithm provides a suitable trade-off between the detection performance and the communication rate.

Xiaoqiang Ren, Jiming Chen, Karl H. Johansson et al.

The problem of quickest change detection with a wireless sensor node is studied in this paper. The sensor that is deployed to monitor the environment has limited energy constraint to the classical quickest change detection problem. We consider the "censoring" strategy at the sensor side, i.e., the sensor selectively sends its observations to the decision maker. The quickest change detection problem is formulated in a minimax way. In particular, our goal is to find the optimal censoring strategy and stopping time such that the detection delay is minimized subject to constraints on both average run length (ARL) and average energy cost before the change. We show that the censoring strategy that has the maximal post-censoring Kullback-Leibler (K-L) divergence coupled with Cumulative Sum (CuSum) and Shiryaev-Roberts-Pollak (SRP) detection procedure is asymptotically optimal for the Lorden's and Pollak's problem as the ARL goes to infinity, respectively. We also show that the asymptotically optimal censoring strategy should use up the available energy and has a very special structure, i.e., the likelihood ratio of the no send region is a single interval, which can be utilized to significantly reduce the computational complexity. Numerical examples are shown to illustrate our results.