Aranya Chakrabortty

Petar Mlinarić, Takayuki Ishizaki, Aranya Chakrabortty et al.

We study nonlinear power systems consisting of generators, generator buses, and non-generator buses. First, looking at a generator and its bus' variables jointly, we introduce a synchronization concept for a pair of such joint generators and buses. We show that this concept is related to graph symmetry. Next, we extend, in two ways, the synchronization from a pair to a partition of all generators in the networks and show that they are related to either graph symmetry or equitable partitions. Finally, we show how an exact reduced model can be obtained by aggregating the generators and associated buses in the network when the original system is synchronized with respect to a partition, provided that the initial condition respects the partition. Additionally, the aggregation-based reduced model is again a power system.

Mang Liao, Aranya Chakrabortty

In this paper we develop a set of algorithms that can detect the identities of malicious data-manipulators in distributed optimization loops for estimating oscillation modes in large power system models. The estimation is posed in terms of a consensus problem among multiple local estimators that jointly solve for the characteristic polynomial of the network model. If any of these local estimates are compromised by a malicious attacker, resulting in an incorrect value of the consensus variable, then the entire estimation loop can be destabilized. We present four iterative algorithms by which this instability can be quickly detected, and the identities of the compromised estimators can be revealed. The algorithms are solely based on the computed values of the estimates, and do not need any information about the model of the power system. Both large and covert attacks are considered. Results are illustrated using simulations of a IEEE 68-bus power system model.

Md Tanvir Arafat Khan, Rafael Cisneros, Aranya Chakrabortty et al.

In this paper a nonlinear control design for power balancing in networked microgrids using energy storage devices is presented. Each microgrid is considered to be interfaced to the distribution feeder though a solid-state transformer (SST). The internal duty cycle based controllers of each SST ensures stable regulation of power commands during normal operation. But problem arises when a sudden change in load or generation occurs in any microgrid in a completely unpredicted way in between the time instants at which the SSTs receive their power setpoints. In such a case, the energy storage unit in that microgrid must produce or absorb the deficit power. The challenge lies in designing a suitable regulator for this purpose owing to the nonlinearity of the battery model and its coupling with the nonlinear SST dynamics. We design an input-output linearization based controller, and show that it guarantees closed-loop stability via a cascade connection with the SST model. The design is also extended to the case when multiple SSTs must coordinate their individual storage controllers to assist a given SST whose storage capacity is insufficient to serve the unpredicted load. The design is verified using the IEEE 34-bus distribution system with nine SST-driven microgrids.

Nan Xue, Aranya Chakrabortty

Wide-area control (WAC) has been shown to be an effective tool for damping low-frequency oscillations in power systems. In the current state of art, WAC is challenged by two main factors - namely, scalability of design and complexity of implementation. In this paper we present a control design called control inversion that bypasses both of these challenges using the idea of clustering. The basic philosophy behind this method is to project the original power system model into a lower-dimensional state-space through clustering and aggregation of generator states, and then designing an LQR controller for the lower-dimensional model. This controller is finally projected back to the original coordinates for wide-area implementation. The main problem is, therefore, posed as finding the projection which best matches the closed-loop performance of the WAC controller with that of a reference LQR controller for damping low-frequency oscillations. We verify the effectiveness of the proposed design using the NPCC 48-machine power system model.

Souvik Chandra, Dhagash Mehta, Aranya Chakrabortty

In this paper we investigate how the equilibrium characteristics of conventional power systems may change with an increase in wind penetration. We first derive a differential-algebraic model of a power system network consisting of synchronous generators, loads and a wind power plant modeled by a wind turbine and a doubly-fed induction generator (DFIG). The models of these three components are coupled via nonlinear power flow equations. In contrast to the traditional approach for solving the power flows via iterative methods that often lead to only local solutions, we apply a recently developed parameter-homotopy based numerical continuation algorithm to compute all possible solutions. The method solves the power flow equations over multiple values of the wind penetration level with far less computational effort instead of solving them at each value individually. We observe that depending on the penetration limit and the setpoint value for the magnitude of the wind bus voltage, the system may exhibit several undesired or even unstable equilibria. We illustrate these results through a detailed simulation of a 5-machine power system model with wind injection, and highlight how the solutions may be helpful for small-signal stability assessment.

Nan Xue, Aranya Chakrabortty

In this paper we present a set of projection-based designs for constructing simplified linear quadratic regulator (LQR) controllers for large-scale network systems. When such systems have tens of thousands of states, the design of conventional LQR controllers becomes numerically challenging, and their implementation requires a large number of communication links. Our proposed algorithms bypass these difficulties by clustering the system states using structural properties of its closed-loop transfer matrix. The assignment of clusters is defined through a structured projection matrix P, which leads to a significantly lower-dimensional LQR design. The reduced-order controller is finally projected back to the original coordinates via an inverse projection. The problem is, therefore, posed as a model matching problem of finding the optimal set of clusters or P that minimizes the H2-norm of the error between the transfer matrix of the full-order network with the full-order LQR and that with the projected LQR. We derive a tractable relaxation for this model matching problem, and design a P that solves the relaxation. The design is shown to be implementable by a convenient, hierarchical two-layer control architecture, requiring far less number of communication links than full-order LQR.

Tomonori Sadamoto, Aranya Chakrabortty, Takayuki Ishizaki et al.

In this paper we address the growing concerns of wind power integration from the perspective of power system dynamics and stability. We propose a new retrofit control technique where an additional controller is designed at the doubly-fed induction generator site inside the wind power plant. This controller cancels the adverse impacts of the power flow from the wind side to the grid side on the dynamics of the overall power system. The main advantage of this controller is that it can be implemented by feeding back only the wind states and wind bus voltage without depending on any of the other synchronous machines in the rest of the system. Through simulations of a 4-machine Kundur power system model we show that the retrofit can efficiently enhance the damping performance of the system variable despite very high values of wind penetration.

Souvik Chandra, Dhagash Mehta, Aranya Chakrabortty

The solution space of any set of power flow equations may contain different number of real-valued solutions. The boundaries that separate these regions are referred to as power flow solution space boundaries. Knowledge of these boundaries is important as they provide a measure for voltage stability. Traditionally, continuation based methods have been employed to compute these boundaries on the basis of initial guesses for the solution. However, with rapid growth of renewable energy sources these boundaries will be increasingly affected by variable parameters such as penetration levels, locations of the renewable sources, and voltage set-points, making it difficult to generate an initial guess that can guarantee all feasible solutions for the power flow problem. In this paper we solve this problem by applying a numerical polynomial homotopy based continuation method. The proposed method guarantees to find all solution boundaries within a given parameter space up to a chosen level of discretization, independent of any initial guess. Power system operators can use this computational tool conveniently to plan the penetration levels of renewable sources at different buses. We illustrate the proposed method through simulations on 3-bus and 10-bus power system examples with renewable generation.

Feier Lian, Aranya Chakrabortty, Alexandra Duel-Hallen

Multi-agent networked linear dynamic systems have attracted attention of researchers in power systems, intelligent transportation, and industrial automation. The agents might cooperatively optimize a global performance objective, resulting in social optimization, or try to satisfy their own selfish objectives using a noncooperative differential game. However, in these solutions, large volumes of data must be sent from system states to possibly distant control inputs, thus resulting in high cost of the underlying communication network. To enable economically-viable communication, a game-theoretic framework is proposed under the \textit{communication cost}, or \textit{sparsity}, constraint, given by the number of communicating state/control input pairs. As this constraint tightens, the system transitions from dense to sparse communication, providing the trade-off between dynamic system performance and information exchange. Moreover, using the proposed sparsity-constrained distributed social optimization and noncooperative game algorithms, we develop a method to allocate the costs of the communication infrastructure fairly and according to the agents' diverse needs for feedback and cooperation. Numerical results illustrate utilization of the proposed algorithms to enable and ensure economic fairness of wide-area control among power companies.

Nan Xue, Aranya Chakrabortty

Standard H2 optimal control of networked dynamic systems tend to become unscalable with network size. Structural constraints can be imposed on the design to counteract this problem albeit at the risk of making the solution non-convex. In this paper, we present a special class of structural constraints such that the H2 design satisfies a quadratic invariance condition, and therefore can be reformulated as a convex problem. This special class consists of structured and weighted projections of the input and output spaces. The choice of these projections can be optimized to match the closed-loop performance of the reformulated controller with that of the standard H2 controller. The advantage is that unlike the latter, the reformulated controller results in a hierarchical implementation which requires significantly lesser number of communication links, while also admitting model and controller reduction that helps the design to scale computationally. We illustrate our design with simulations of a 500-node network.

Mohammad Zakaria Haider, Amit Kumar Podder, Prabin Mali et al.

The rapid deployment of electric vehicle charging stations (EVCS) within distribution networks necessitates intelligent and adaptive control to maintain the grid's resilience and reliability. In this work, we propose PHANTOM, a physics-aware adversarial network that is trained and optimized through a multi-agent reinforcement learning model. PHANTOM integrates a physics-informed neural network (PINN) enabled by federated learning (FL) that functions as a digital twin of EVCS-integrated systems, ensuring physically consistent modeling of operational dynamics and constraints. Building on this digital twin, we construct a multi-agent RL environment that utilizes deep Q-networks (DQN) and soft actor-critic (SAC) methods to derive adversarial false data injection (FDI) strategies capable of bypassing conventional detection mechanisms. To examine the broader grid-level consequences, a transmission and distribution (T and D) dual simulation platform is developed, allowing us to capture cascading interactions between EVCS disturbances at the distribution level and the operations of the bulk transmission system. Results demonstrate how learned attack policies disrupt load balancing and induce voltage instabilities that propagate across T and D boundaries. These findings highlight the critical need for physics-aware cybersecurity to ensure the resilience of large-scale vehicle-grid integration.

Gangshan Jing, He Bai, Jemin George et al.

Achieving distributed reinforcement learning (RL) for large-scale cooperative multi-agent systems (MASs) is challenging because: (i) each agent has access to only limited information; (ii) issues on convergence or computational complexity emerge due to the curse of dimensionality. In this paper, we propose a general computationally efficient distributed framework for cooperative multi-agent reinforcement learning (MARL) by utilizing the structures of graphs involved in this problem. We introduce three coupling graphs describing three types of inter-agent couplings in MARL, namely, the state graph, the observation graph and the reward graph. By further considering a communication graph, we propose two distributed RL approaches based on local value-functions derived from the coupling graphs. The first approach is able to reduce sample complexity significantly under specific conditions on the aforementioned four graphs. The second approach provides an approximate solution and can be efficient even for problems with dense coupling graphs. Here there is a trade-off between minimizing the approximation error and reducing the computational complexity. Simulations show that our RL algorithms have a significantly improved scalability to large-scale MASs compared with centralized and consensus-based distributed RL algorithms.

Gangshan Jing, He Bai, Jemin George et al.

Existing distributed cooperative multi-agent reinforcement learning (MARL) frameworks usually assume undirected coordination graphs and communication graphs while estimating a global reward via consensus algorithms for policy evaluation. Such a framework may induce expensive communication costs and exhibit poor scalability due to requirement of global consensus. In this work, we study MARLs with directed coordination graphs, and propose a distributed RL algorithm where the local policy evaluations are based on local value functions. The local value function of each agent is obtained by local communication with its neighbors through a directed learning-induced communication graph, without using any consensus algorithm. A zeroth-order optimization (ZOO) approach based on parameter perturbation is employed to achieve gradient estimation. By comparing with existing ZOO-based RL algorithms, we show that our proposed distributed RL algorithm guarantees high scalability. A distributed resource allocation example is shown to illustrate the effectiveness of our algorithm.

Gangshan Jing, He Bai, Jemin George et al.

Recently introduced distributed zeroth-order optimization (ZOO) algorithms have shown their utility in distributed reinforcement learning (RL). Unfortunately, in the gradient estimation process, almost all of them require random samples with the same dimension as the global variable and/or require evaluation of the global cost function, which may induce high estimation variance for large-scale networks. In this paper, we propose a novel distributed zeroth-order algorithm by leveraging the network structure inherent in the optimization objective, which allows each agent to estimate its local gradient by local cost evaluation independently, without use of any consensus protocol. The proposed algorithm exhibits an asynchronous update scheme, and is designed for stochastic non-convex optimization with a possibly non-convex feasible domain based on the block coordinate descent method. The algorithm is later employed as a distributed model-free RL algorithm for distributed linear quadratic regulator design, where a learning graph is designed to describe the required interaction relationship among agents in distributed learning. We provide an empirical validation of the proposed algorithm to benchmark its performance on convergence rate and variance against a centralized ZOO algorithm.

Gangshan Jing, He Bai, Jemin George et al.

Individual agents in a multi-agent system (MAS) may have decoupled open-loop dynamics, but a cooperative control objective usually results in coupled closed-loop dynamics thereby making the control design computationally expensive. The computation time becomes even higher when a learning strategy such as reinforcement learning (RL) needs to be applied to deal with the situation when the agents dynamics are not known. To resolve this problem, we propose a parallel RL scheme for a linear quadratic regulator (LQR) design in a continuous-time linear MAS. The idea is to exploit the structural properties of two graphs embedded in the $Q$ and $R$ weighting matrices in the LQR objective to define an orthogonal transformation that can convert the original LQR design to multiple decoupled smaller-sized LQR designs. We show that if the MAS is homogeneous then this decomposition retains closed-loop optimality. Conditions for decomposability, an algorithm for constructing the transformation matrix, a parallel RL algorithm, and robustness analysis when the design is applied to non-homogeneous MAS are presented. Simulations show that the proposed approach can guarantee significant speed-up in learning without any loss in the cumulative value of the LQR cost.

Sayak Mukherjee, He Bai, Aranya Chakrabortty

We present a set of model-free, reduced-dimensional reinforcement learning (RL) based optimal control designs for linear time-invariant singularly perturbed (SP) systems. We first present a state-feedback and output-feedback based RL control design for a generic SP system with unknown state and input matrices. We take advantage of the underlying time-scale separation property of the plant to learn a linear quadratic regulator (LQR) for only its slow dynamics, thereby saving a significant amount of learning time compared to the conventional full-dimensional RL controller. We analyze the sub-optimality of the design using SP approximation theorems and provide sufficient conditions for closed-loop stability. Thereafter, we extend both designs to clustered multi-agent consensus networks, where the SP property reflects through clustering. We develop both centralized and cluster-wise block-decentralized RL controllers for such networks, in reduced dimensions. We demonstrate the details of the implementation of these controllers using simulations of relevant numerical examples and compare them with conventional RL designs to show the computational benefits of our approach.

Nandini Negi, Aranya Chakrabortty

Recent years have seen several new directions in the design of sparse control of cyber-physical systems (CPSs) driven by the objective of reducing communication cost. One common assumption made in these designs is that the communication happens over a dedicated network. For many practical applications, however, communication must occur over shared networks, leading to two critical design challenges, namely - time-delays in the feedback and fair sharing of bandwidth among users. In this paper, we present a set of sparse H2 control designs under these two design constraints. An important aspect of our design is that the delay itself can be a function of sparsity, which leads to an interesting pattern of trade-offs in the H2 performance. We present three distinct algorithms. The first algorithm preconditions the assignable bandwidth to the network and produces an initial guess for a stabilizing controller. This is followed by our second algorithm, which sparsifies this controller while simultaneously adapting the feedback delay and optimizing the H2 performance using alternating directions method of multipliers (ADMM). The third algorithm extends this approach to a multiple user scenario where optimal number of communication links, whose total sum is fixed, is distributed fairly among users by minimizing the variance of their H2 performances. The problem is cast as a difference-of-convex (DC) program with mixed-integer linear program (MILP) constraints. We provide theorems to prove convergence of these algorithms, followed by validation through numerical simulations.

Nilanjan Roy Chowdhury, Nandini Negi, Aranya Chakrabortty

This paper presents a new counter-measure to mitigate denial-of-service cyber-attacks in linear time-invariant (LTI) systems. We first design a sparse linear quadratic regulator (LQR) optimal controller for a given LTI plant and evaluate the priority of the feedback communication links in terms of the loss of closed-loop performance when the corresponding block of the feedback gain matrix is removed. An attacker may know about this priority ordering, and thereby attack the links with the highest priority. To prevent this, we present a message rerouting strategy by which the states that are scheduled to be transmitted through the high priority links can be rerouted through lower priority ones in case the former get attacked. Since the attacked link is not available for service, and the states of the low priority links can no longer be accommodated either, we run a structured $\mathcal{H}_2$ control algorithm to determine the post-attack optimal feedback gains. We illustrate various aspects of the proposed algorithms by simulations.

Amirhassan Fallah Dizche, Aranya Chakrabortty, Alexandra Duel-Hallen

In this paper we present an online wide-area oscillation damping control (WAC) design for uncertain models of power systems using ideas from reinforcement learning. We assume that the exact small-signal model of the power system at the onset of a contingency is not known to the operator and use the nominal model and online measurements of the generator states and control inputs to rapidly converge to a state-feedback controller that minimizes a given quadratic energy cost. However, unlike conventional linear quadratic regulators (LQR), we intend our controller to be sparse, so its implementation reduces the communication costs. We, therefore, employ the gradient support pursuit (GraSP) optimization algorithm to impose sparsity constraints on the control gain matrix during learning. The sparse controller is thereafter implemented using distributed communication. Using the IEEE 39-bus power system model with 1149 unknown parameters, it is demonstrated that the proposed learning method provides reliable LQR performance while the controller matched to the nominal model becomes unstable for severely uncertain systems.

Pratishtha Shukla, Aranya Chakrabortty, Alexandra Duel-Hallen

We formulate a resource-planning game between an attacker and a defender of a network control system. We consider the network to be operating in closed-loop with a linear quadratic regulator (LQR). We construct a general-sum, two-player, mixed strategy game, where the attacker attempts to destroy communication equipment of some nodes, and thereby render the LQR feedback gain matrix to be sparse, leading to degradation of closed-loop performance. The defender, on the other hand, aims to prevent this loss. Both players trade their control performance objectives for the cost of their actions. A Mixed Strategy Nash Equilibrium (MSNE) of the game represents the allocation of the players' respective resources for attacking or protecting the network nodes. We analyze the dependence of a MSNE on the relative budgets of the players as well as on the important network nodes that must be preserved to achieve a desirable LQR performance. MSNE is computed using nonlinear programming. Results are validated using the New England power system model, and it is shown that reliable defense is feasible unless the cost of attack is very low or much smaller than the cost of protection per generator.

Tomonori Sadamoto, Aranya Chakrabortty, Takayuki Ishizaki et al.

This article presents a suite of new control designs for next-generation electric smart grids. The future grid will consist of thousands of non-conventional renewable generation sources such as wind, solar, and energy storage. These new components are collectively referred to as distributed energy resources (DER). The article presents a comprehensive list of dynamic models for DERs, and shows their coupling with the conventional generators and loads. It then presents several innovative control designs that can be used for facilitating large-scale DER integration. Ideas from decentralized retrofit control and distributed sparsity-promoting optimal control are used for developing these designs, followed by illustrations on an IEEE power system test model.

Seyedbehzad Nabavi, Aranya Chakrabortty, Pramod P. Khargonekar

In this paper we present a sufficient condition that guarantees identifiability of linear network dynamic systems exhibiting continuous-time weighted consensus protocols with acyclic structure. Each edge of the underlying network graph $\mathcal G$ of the system is defined by a constant parameter, referred to as the weight of the edge, while each node is defined by a scalar state whose dynamics evolve as the weighted linear combination of its difference with the states of its neighboring nodes. Following the classical definitions of identifiability and indistinguishability, we first derive a condition that ensure the identifiability of the edge weights of $\mathcal G$ in terms of the associated transfer function. Using this characterization, we propose a sensor placement algorithm that guarantees identifiability of the edge weights. We describe our results using several illustrative examples.