Husheng Li

Rukun Mao, Husheng Li

The increasing availability of distributed energy resources (DERs) and sensors in smart grid, as well as overlaying communication network, provides substantial potential benefits for improving the power system's reliability. In this paper, the problem of sensor selection is studied for the MAC layer design of wireless sensor networks for regulating the voltages in smart grid. The framework of hybrid dynamical system is proposed, using Kalman filter for voltage state estimation and LQR feedback control for voltage adjustment. The approach to obtain the optimal sensor selection sequence is studied. A sub- optimal sequence is obtained by applying the sliding window algorithm. Simulation results show that the proposed sensor selection strategy achieves a 40% performance gain over the baseline algorithm of the round-robin sensor polling.

Husheng Li, Lifeng Lai, Seddik M. Djouadi

Smart grid, equipped with modern communication infrastructures, is subject to possible cyber attacks. Particularly, false report attacks which replace the sensor reports with fraud ones may cause the instability of the whole power grid or even result in a large area blackout. In this paper, a trustiness system is introduced to the controller, who computes the trustiness of different sensors by comparing its prediction, obtained from Kalman filtering, on the system state with the reports from sensor. The trustiness mechanism is discussed and analyzed for the Linear Quadratic Regulation (LQR) controller. Numerical simulations show that the trustiness system can effectively combat the cyber attacks to smart grid.

Xuefeng Wang, Henglin Pu, Hyung Jun Kim et al.

Safe Multi-agent reinforcement learning (safe MARL) has increasingly gained attention in recent years, emphasizing the need for agents to not only optimize the global return but also adhere to safety requirements through behavioral constraints. Some recent work has integrated control theory with multi-agent reinforcement learning to address the challenge of ensuring safety. However, there have been only very limited applications of Model Predictive Control (MPC) methods in this domain, primarily due to the complex and implicit dynamics characteristic of multi-agent environments. To bridge this gap, we propose a novel method called Deep Learning-Based Model Predictive Control for Safe Multi-Agent Reinforcement Learning (DeepSafeMPC). The key insight of DeepSafeMPC is leveraging a entralized deep learning model to well predict environmental dynamics. Our method applies MARL principles to search for optimal solutions. Through the employment of MPC, the actions of agents can be restricted within safe states concurrently. We demonstrate the effectiveness of our approach using the Safe Multi-agent MuJoCo environment, showcasing significant advancements in addressing safety concerns in MARL.

Xuefeng Wang, Lei Zhang, Henglin Pu et al.

Existing reinforcement learning (RL) methods struggle with complex dynamical systems that demand interactions at high frequencies or irregular time intervals. Continuous-time RL (CTRL) has emerged as a promising alternative by replacing discrete-time Bellman recursion with differential value functions defined as viscosity solutions of the Hamilton--Jacobi--Bellman (HJB) equation. While CTRL has shown promise, its applications have been largely limited to the single-agent domain. This limitation stems from two key challenges: (i) conventional solution methods for HJB equations suffer from the curse of dimensionality (CoD), making them intractable in high-dimensional systems; and (ii) even with HJB-based learning approaches, accurately approximating centralized value functions in multi-agent settings remains difficult, which in turn destabilizes policy training. In this paper, we propose a CT-MARL framework that uses physics-informed neural networks (PINNs) to approximate HJB-based value functions at scale. To ensure the value is consistent with its differential structure, we align value learning with value-gradient learning by introducing a Value Gradient Iteration (VGI) module that iteratively refines value gradients along trajectories. This improves gradient fidelity, in turn yielding more accurate values and stronger policy learning. We evaluate our method using continuous-time variants of standard benchmarks, including multi-agent particle environment (MPE) and multi-agent MuJoCo. Our results demonstrate that our approach consistently outperforms existing continuous-time RL baselines and scales to complex multi-agent dynamics.

Chen Zhang, Husheng Li, Xiang Liu et al.

Recent advances in personalized federated learning have focused on addressing client model heterogeneity. However, most existing methods still require external data, rely on model decoupling, or adopt partial learning strategies, which can limit their practicality and scalability. In this paper, we revisit hypernetwork-based methods and leverage their strong generalization capabilities to design a simple yet effective framework for heterogeneous personalized federated learning. Specifically, we propose MH-pFedHN, which leverages a server-side hypernetwork that takes client-specific embedding vectors as input and outputs personalized parameters tailored to each client's heterogeneous model. To promote knowledge sharing and reduce computation, we introduce a multi-head structure within the hypernetwork, allowing clients with similar model sizes to share heads. Furthermore, we further propose MH-pFedHNGD, which integrates an optional lightweight global model to improve generalization. Our framework does not rely on external datasets and does not require disclosure of client model architectures, thereby offering enhanced privacy and flexibility. Extensive experiments on multiple benchmarks and model settings demonstrate that our approach achieves competitive accuracy, strong generalization, and serves as a robust baseline for future research in model-heterogeneous personalized federated learning.

Lei Chu, Ling Pei, Husheng Li et al.

This paper develops a new deep neural network optimized equalization framework for massive multiple input multiple output orthogonal frequency division multiplexing (MIMOOFDM) systems that employ low-resolution analog-to-digital converters (ADCs) at the base station (BS). The use of lowresolution ADCs could largely reduce hardware complexity and circuit power consumption, however, it makes the channel station information almost blind to the BS, hence causing difficulty in solving the equalization problem. In this paper, we consider a supervised learning architecture, where the goal is to learn a representative function that can predict the targets (constellation points) from the inputs (outputs of the low-resolution ADCs) based on the labeled training data (pilot signals). Especially, our main contributions are two-fold: 1) First, we design a new activation function, whose outputs are close to the constellation points when the parameters are finally optimized, to help us fully exploit the stochastic gradient descent method for the discrete optimization problem. 2) Second, an unsupervised loss is designed and then added to the optimization objective, aiming to enhance the representation ability (so-called generalization). Lastly, various experimental results confirm the superiority of the proposed equalizer over some existing ones, particularly when the statistics of the channel state information are unclear.

Husheng Li

The theoretical explanation for deep neural network (DNN) is still an open problem. In this paper DNN is considered as a discrete-time dynamical system due to its layered structure. The complexity provided by the nonlinearity in the dynamics is analyzed in terms of topological entropy and chaos characterized by Lyapunov exponents. The properties revealed for the dynamics of DNN are applied to analyze the corresponding capabilities of classification and generalization.

Zi Wang, Dali Wang, Chengcheng Li et al.

Cell movement in the early phase of C. elegans development is regulated by a highly complex process in which a set of rules and connections are formulated at distinct scales. Previous efforts have shown that agent-based, multi-scale modeling systems can integrate physical and biological rules and provide new avenues to study developmental systems. However, the application of these systems to model cell movement is still challenging and requires a comprehensive understanding of regulation networks at the right scales. Recent developments in deep learning and reinforcement learning provide an unprecedented opportunity to explore cell movement using 3D time-lapse images. We present a deep reinforcement learning approach within an ABM system to characterize cell movement in C. elegans embryogenesis. Our modeling system captures the complexity of cell movement patterns in the embryo and overcomes the local optimization problem encountered by traditional rule-based, ABM that uses greedy algorithms. We tested our model with two real developmental processes: the anterior movement of the Cpaaa cell via intercalation and the rearrangement of the left-right asymmetry. In the first case, model results showed that Cpaaa's intercalation is an active directional cell movement caused by the continuous effects from a longer distance, as opposed to a passive movement caused by neighbor cell movements. This is because the learning-based simulation found that a passive movement model could not lead Cpaaa to the predefined destination. In the second case, a leader-follower mechanism well explained the collective cell movement pattern. These results showed that our approach to introduce deep reinforcement learning into ABM can test regulatory mechanisms by exploring cell migration paths in a reverse engineering perspective. This model opens new doors to explore large datasets generated by live imaging.

Zhenghao Zhang, Matthew Trinkle, Aleksandar D. Dimitrovski et al.

A novel time synchronization attack (TSA) on wide area monitoring systems in smart grid has been identified in the first part of this paper. A cross layer detection mechanism is proposed to combat TSA in part II of this paper. In the physical layer, we propose a GPS carrier signal noise ratio (C/No) based spoofing detection technique. In addition, a patch-monopole hybrid antenna is applied to receive GPS signal. By computing the standard deviation of the C/No difference from two GPS receivers, a priori probability of spoofing detection is fed to the upper layer, where power system state is estimated and controlled. A trustworthiness based evaluation method is applied to identify the PMU being under TSA. Both the physical layer and upper layer algorithms are integrated to detect the TSA, thus forming a cross layer mechanism. Experiment is carried out to verify the effectiveness of the proposed TSA detection algorithm.

Zhenghao Zhang, Shuping Gong, Aleksandar D. Dimitrovski et al.

Many operations in power grids, such as fault detection and event location estimation, depend on precise timing information. In this paper, a novel Time Synchronization Attack (TSA) is proposed to attack the timing information in smart grid. Since many applications in smart grid utilize synchronous measurements and most of the measurement devices are equipped with global positioning system (GPS) for precise timing, it is highly probable to attack the measurement system by spoofing the GPS. The effectiveness of TSA is demonstrated for three applications of phasor measurement unit (PMU) in smart grid, namely transmission line fault detection, voltage stability monitoring and event locationing. The validity of TSA is demonstrated by numerical simulations.

Shuping Gong, Zhenghao Zhang, Husheng Li et al.

Many operations in power grids, such as fault detection and event location estimation, depend on precise timing information. In this paper, a novel time stamp attack (TSA) is proposed to attack the timing information in smart grid. Since many applications in smart grid utilize synchronous measurements and most of the measurement devices are equipped with global positioning system (GPS) for precise timing, it is highly probable to attack the measurement system by spoofing the GPS. The effectiveness of TSA is demonstrated for three applications of phasor measurement unit (PMU) in smart grid, namely transmission line fault detection, voltage stability monitoring and event locationing.