Deepjyoti Deka

Mile Mitrovic, Ognjen Kundacina, Aleksandr Lukashevich et al.

The Gaussian Process (GP) based Chance-Constrained Optimal Power Flow (CC-OPF) is an open-source Python code developed for solving economic dispatch (ED) problem in modern power grids. In recent years, integrating a significant amount of renewables into a power grid causes high fluctuations and thus brings a lot of uncertainty to power grid operations. This fact makes the conventional model-based CC-OPF problem non-convex and computationally complex to solve. The developed tool presents a novel data-driven approach based on the GP regression model for solving the CC-OPF problem with a trade-off between complexity and accuracy. The proposed approach and developed software can help system operators to effectively perform ED optimization in the presence of large uncertainties in the power grid.

Deepjyoti Deka, Scott Backhaus, Michael Chertkov

Distribution grids represent the final tier in electric networks consisting of medium and low voltage lines that connect the distribution substations to the end-users. Traditionally, distribution networks have been operated in a radial topology that may be changed from time to time. Due to absence of a significant number of real-time line monitoring devices in the distribution grid, estimation of the topology is a problem critical for its observability and control. This paper develops a novel graphical learning based approach to estimate the radial operational grid structure using voltage measurements collected from the grid loads. The learning algorithm is based on conditional independence tests for continuous variables over chordal graphs and has wide applicability. It is proven that the scheme can be used for several power flow laws (DC or AC approximations) and more importantly is independent of the specific probability distribution controlling individual bus power usage. The complexity of the algorithm is discussed and its performance is demonstrated by simulations on distribution test cases.

Sejun Park, Deepjyoti Deka, Michael Chertkov

Limited presence of nodal and line meters in distribution grids hinders their optimal operation and participation in real-time markets. In particular lack of real-time information on the grid topology and infrequently calibrated line parameters (impedances) adversely affect the accuracy of any operational power flow control. This paper suggests a novel algorithm for learning the topology of distribution grid and estimating impedances of the operational lines with minimal observational requirements - it provably reconstructs topology and impedances using voltage and injection measured only at the terminal (end-user) nodes of the distribution grid. All other (intermediate) nodes in the network may be unobserved/hidden. Furthermore no additional input (e.g., number of grid nodes, historical information on injections at hidden nodes) is needed for the learning to succeed. Performance of the algorithm is illustrated in numerical experiments on the IEEE and custom power distribution models.

Deepjyoti Deka, Saurav Talukdar, Michael Chertkov et al.

Graphical models are a succinct way to represent the structure in probability distributions. This article analyzes the graphical model of nodal voltages in non-radial power distribution grids. Using algebraic and structural properties of graphical models, algorithms exactly determining topology and detecting line changes for distribution grids are presented along with their theoretical limitations. We show that if distribution grids have cycles/loops of size greater than three, then nodal voltages are sufficient for efficient topology estimation without additional assumptions on system parameters. In contrast, line failure or change detection using nodal voltages does not require any structural assumption. Under noisy measurements, we provide the first non-trivial bounds on the maximum noise that the system can tolerate for asymptotically correct topology recovery. The performance of the designed algorithms is validated with nonlinear AC power flow samples generated by Matpower on test grids, including scenarios with injection correlations and system noise.

Ali Hassan, Robert Mieth, Michael Chertkov et al.

Distribution system operators (DSO) world-wide foresee a rapid roll-out of distributed energy resources. From the system perspective, their reliable and cost effective integration requires accounting for their physical properties in operating tools used by the DSO. This paper describes an approach to leverage the dispatch flexibility of thermostatically controlled loads (TCLs) for operating distribution systems with a high penetration level of photovoltaic resources. Each TCL ensemble is modeled using the Markov Decision Process (MDP). The MDP model is then integrated with the chance-constrained optimal power flow that accounts for the uncertainty of PV resources. Since the integrated optimization model cannot be solved efficiently by existing dynamic programming methods or off-the-shelf solvers, this paper proposes an iterative Spatio-Temporal Dual Decomposition algorithm (ST-D2). We demonstrate the usefulness of the proposed integrated optimization and ST-D2 algorithm on the IEEE 33-bus test system.

Sejun Park, Deepjyoti Deka, Scott Backhaus et al.

Efficient operation of distribution grids in the smart-grid era is hindered by the limited presence of real-time nodal and line meters. In particular, this prevents the easy estimation of grid topology and associated line parameters that are necessary for control and optimization efforts in the grid. This paper studies the problems of topology and parameter estimation in radial balanced distribution grids where measurements are restricted to only the leaf nodes and all intermediate nodes are unobserved/hidden. To this end, we propose two exact learning algorithms that use balanced voltage and injection measured only at the end-users. The first algorithm requires time-stamped voltage samples, statistics of nodal power injections and permissible line impedances to recover the true topology. The second and improved algorithm requires only time-stamped voltage and complex power samples to recover both the true topology and impedances without any additional input (e.g., number of grid nodes, statistics of injections at hidden nodes, permissible line impedances). We prove the correctness of both learning algorithms for grids where unobserved buses/nodes have a degree greater than three and discuss extensions to regimes where that assumption doesn't hold. Further, we present computational and, more importantly, the sample complexity of our proposed algorithm for joint topology and impedance estimation. We illustrate the performance of the designed algorithms through numerical experiments on the IEEE and custom power distribution models.

Deepjyoti Deka, Scott Backhaus, Michael Chertkov

Traditionally power distribution networks are either not observable or only partially observable. This complicates development and implementation of new smart grid technologies, such as those related to demand response, outage detection and management, and improved load-monitoring. In this two part paper, inspired by proliferation of metering technology, we discuss estimation problems in structurally loopy but operationally radial distribution grids from measurements, e.g. voltage data, which are either already available or can be made available with a relatively minor investment. In Part I, the objective is to learn the operational layout of the grid. Part II of this paper presents algorithms that estimate load statistics or line parameters in addition to learning the grid structure. Further, Part II discusses the problem of structure estimation for systems with incomplete measurement sets. Our newly suggested algorithms apply to a wide range of realistic scenarios. The algorithms are also computationally efficient -- polynomial in time -- which is proven theoretically and illustrated computationally on a number of test cases. The technique developed can be applied to detect line failures in real time as well as to understand the scope of possible adversarial attacks on the grid.

Michael Chertkov, Vladimir Y. Chernyak, Deepjyoti Deka

A Markov decision process (MDP) framework is adopted to represent ensemble control of devices with cyclic energy consumption patterns, e.g., thermostatically controlled loads. Specifically we utilize and develop the class of MDP models previously coined linearly solvable MDPs, that describe optimal dynamics of the probability distribution of an ensemble of many cycling devices. Two principally different settings are discussed. First, we consider optimal strategy of the ensemble aggregator balancing between minimization of the cost of operations and minimization of the ensemble welfare penalty, where the latter is represented as a KL-divergence between actual and normal probability distributions of the ensemble. Then, second, we shift to the demand response setting modeling the aggregator's task to minimize the welfare penalty under the condition that the aggregated consumption matches the targeted time-varying consumption requested by the system operator. We discuss a modification of both settings aimed at encouraging or constraining the transitions between different states. The dynamic programming feature of the resulting modified MDPs is always preserved; however, `linear solvability' is lost fully or partially, depending on the type of modification. We also conducted some (limited in scope) numerical experimentation using the formulations of the first setting. We conclude by discussing future generalizations and applications.

Md Umar Hashmi, Deepjyoti Deka, Ana Busic et al.

The importance of reactive power compensation for power factor (PF) correction will significantly increase with the large-scale integration of distributed generation interfaced via inverters producing only active power. In this work, we focus on co-optimizing energy storage for performing energy arbitrage as well as local power factor correction. The joint optimization problem is non-convex, but can be solved efficiently using a McCormick relaxation along with penalty-based schemes. Using numerical simulations on real data and realistic storage profiles, we show that energy storage can correct PF locally without reducing arbitrage profit. It is observed that active and reactive power control is largely decoupled in nature for performing arbitrage and PF correction (PFC). Furthermore, we consider a real-time implementation of the problem with uncertain load, renewable and pricing profiles. We develop a model predictive control based storage control policy using auto-regressive forecast for the uncertainty. We observe that PFC is primarily governed by the size of the converter and therefore, look-ahead in time in the online setting does not affect PFC noticeably. However, arbitrage profit are more sensitive to uncertainty for batteries with faster ramp rates compared to slow ramping batteries.

Deepjyoti Deka, Scott Backhaus, Michael Chertkov

Distribution grids refer to the part of the power grid that delivers electricity from substations to the loads. Structurally a distribution grid is operated in one of several radial/tree-like topologies that are derived from an original loopy grid graph by opening switches on some lines. Due to limited presence of real-time switch monitoring devices, the operating structure needs to be estimated indirectly. This paper presents a new learning algorithm that uses only nodal voltage measurements to determine the operational radial structure. The algorithm is based on the key result stating that the correct operating structure is the optimal solution of the minimum-weight spanning tree problem over the original loopy graph where weights on all permissible edges/lines (open or closed) is the variance of nodal voltage difference at the edge ends. Compared to existing work, this spanning tree based approach has significantly lower complexity as it does not require information on line parameters. Further, a modified learning algorithm is developed for cases when the input voltage measurements are limited to only a subset of the total grid nodes. Performance of the algorithms (with and without missing data) is demonstrated by experiments on test cases.

Ali Hassan, Yury Dvorkin, Deepjyoti Deka et al.

Distribution systems are undergoing a dramatic transition from a passive circuit that routinely disseminates electric power among downstream nodes to the system with distributed energy resources. The distributed energy resources come in a variety of technologies and typically include photovoltaic (PV) arrays, thermostatically controlled loads, energy storage units. Often these resources are interfaced with the system via inverters that can adjust active and reactive power injections, thus supporting the operational performance of the system. This paper designs a control policy for such inverters using the local power flow measurements. The control actuates active and reactive power injections of the inverter-based distributed energy resources. This strategy is then incorporated into a chance-constrained, decentralized optimal power flow formulation to maintain voltage levels and power flows within their limits and to mitigate the volatility of (PV) resources.

Deepjyoti Deka, Sidhant Misra

The optimal power flow is an optimization problem used in power systems operational planning to maximize economic efficiency while satisfying demand and maintaining safety margins. Due to uncertainty and variability in renewable energy generation and demand, the optimal solution needs to be updated in response to observed uncertainty realizations or near real-time forecast updates. To address the challenge of computing such frequent real-time updates to the optimal solution, recent literature has proposed the use of machine learning to learn the mapping between the uncertainty realization and the optimal solution. Further, learning the active set of constraints at optimality, as opposed to directly learning the optimal solution, has been shown to significantly simplify the machine learning task, and the learnt model can be used to predict optimal solutions in real-time. In this paper, we propose the use of classification algorithms to learn the mapping between the uncertainty realization and the active set of constraints at optimality, thus further enhancing the computational efficiency of the real-time prediction. We employ neural net classifiers for this task and demonstrate the excellent performance of this approach on a number of systems in the IEEE PES PGLib-OPF benchmark library.

Deepjyoti Deka, Vassilis Kekatos, Guido Cavraro

Unveiling feeder topologies from data is of paramount importance to advance situational awareness and proper utilization of smart resources in power distribution grids. This tutorial summarizes, contrasts, and establishes useful links between recent works on topology identification and detection schemes that have been proposed for power distribution grids. The primary focus is to highlight methods that overcome the limited availability of measurement devices in distribution grids, while enhancing topology estimates using conservation laws of power-flow physics and structural properties of feeders. Grid data from phasor measurement units or smart meters can be collected either passively in the traditional way, or actively, upon actuating grid resources and measuring the feeder's voltage response. Analytical claims on feeder identifiability and detectability are reviewed under disparate meter placement scenarios. Such topology learning claims can be attained exactly or approximately so via algorithmic solutions with various levels of computational complexity, ranging from least-squares fits to convex optimization problems, and from polynomial-time searches over graphs to mixed-integer programs. Although the emphasis is on radial single-phase feeders, extensions to meshed and/or multiphase circuits are sometimes possible and discussed. This tutorial aspires to provide researchers and engineers with knowledge of the current state-of-the-art in tractable distribution grid learning and insights into future directions of work.

Michael Chertkov, Deepjyoti Deka, Yury Dvorkin

Flexible loads, e.g. thermostatically controlled loads (TCLs), are technically feasible to participate in demand response (DR) programs. On the other hand, there is a number of challenges that need to be resolved before it can be implemented in practice en masse. First, individual TCLs must be aggregated and operated in sync to scale DR benefits. Second, the uncertainty of TCLs needs to be accounted for. Third, exercising the flexibility of TCLs needs to be coordinated with distribution system operations to avoid unnecessary power losses and compliance with power flow and voltage limits. This paper addresses these challenges. We propose a network-constrained, open-loop, stochastic optimal control formulation. The first part of this formulation represents ensembles of collocated TCLs modelled by an aggregated Markov Process (MP), where each MP state is associated with a given power consumption or production level. The second part extends MPs to a multi-period distribution power flow optimization. In this optimization, the control of TCL ensembles is regulated by transition probability matrices and physically enabled by local active and reactive power controls at TCL locations. The optimization is solved with a Spatio-Temporal Dual Decomposition (ST-D2) algorithm. The performance of the proposed formulation and algorithm is demonstrated on the IEEE 33-bus distribution model.

Yize Chen, Yushi Tan, Deepjyoti Deka

Recent advances in Machine Learning(ML) have led to its broad adoption in a series of power system applications, ranging from meter data analytics, renewable/load/price forecasting to grid security assessment. Although these data-driven methods yield state-of-the-art performances in many tasks, the robustness and security of applying such algorithms in modern power grids have not been discussed. In this paper, we attempt to address the issues regarding the security of ML applications in power systems. We first show that most of the current ML algorithms proposed in power systems are vulnerable to adversarial examples, which are maliciously crafted input data. We then adopt and extend a simple yet efficient algorithm for finding subtle perturbations, which could be used for generating adversaries for both categorical(e.g., user load profile classification) and sequential applications(e.g., renewables generation forecasting). Case studies on classification of power quality disturbances and forecast of building loads demonstrate the vulnerabilities of current ML algorithms in power networks under our adversarial designs. These vulnerabilities call for design of robust and secure ML algorithms for real world applications.

Deepjyoti Deka, Michael Chertkov, Scott Backhaus

Optimal operation of distribution grid resources relies on accurate estimation of its state and topology. Practical estimation of such quantities is complicated by the limited presence of real-time meters. This paper discusses a theoretical framework to jointly estimate the operational topology and statistics of injections in radial distribution grids under limited availability of nodal voltage measurements. In particular we show that our proposed algorithms are able to provably learn the exact grid topology and injection statistics at all unobserved nodes as long as they are not adjacent. The algorithm design is based on novel ordered trends in voltage magnitude fluctuations at node groups, that are independently of interest for radial physical flow networks. The complexity of the designed algorithms is theoretically analyzed and their performance validated using both linearized and non-linear AC power flow samples in test distribution grids.

Deepjyoti Deka, Harsha Nagarajan, Scott Backhaus

The transient response of power grids to external disturbances influences their stable operation. This paper studies the effect of topology in linear time-invariant dynamics of different power grids. For a variety of objective functions, a unified framework based on $H_2$ norm is presented to analyze the robustness to ambient fluctuations. Such objectives include loss reduction, weighted consensus of phase angle deviations, oscillations in nodal frequency, and other graphical metrics. The framework is then used to study the problem of optimal topology design for robust control goals of different grids. For radial grids, the problem is shown as equivalent to the hard "optimum communication spanning tree" problem in graph theory and a combinatorial topology construction is presented with bounded approximation gap. Extended to loopy (meshed) grids, a greedy topology design algorithm is discussed. The performance of the topology design algorithms under multiple control objectives are presented on both loopy and radial test grids. Overall, this paper analyzes topology design algorithms on a broad class of control problems in power grid by exploring their combinatorial and graphical properties.

Mile Mitrovic, Aleksandr Lukashevich, Petr Vorobev et al.

In recent years, electricity generation has been responsible for more than a quarter of the greenhouse gas emissions in the US. Integrating a significant amount of renewables into a power grid is probably the most accessible way to reduce carbon emissions from power grids and slow down climate change. Unfortunately, the most accessible renewable power sources, such as wind and solar, are highly fluctuating and thus bring a lot of uncertainty to power grid operations and challenge existing optimization and control policies. The chance-constrained alternating current (AC) optimal power flow (OPF) framework finds the minimum cost generation dispatch maintaining the power grid operations within security limits with a prescribed probability. Unfortunately, the AC-OPF problem's chance-constrained extension is non-convex, computationally challenging, and requires knowledge of system parameters and additional assumptions on the behavior of renewable distribution. Known linear and convex approximations to the above problems, though tractable, are too conservative for operational practice and do not consider uncertainty in system parameters. This paper presents an alternative data-driven approach based on Gaussian process (GP) regression to close this gap. The GP approach learns a simple yet non-convex data-driven approximation to the AC power flow equations that can incorporate uncertainty inputs. The latter is then used to determine the solution of CC-OPF efficiently, by accounting for both input and parameter uncertainty. The practical efficiency of the proposed approach using different approximations for GP-uncertainty propagation is illustrated over numerous IEEE test cases.

Saurav Talukdar, Deepjyoti Deka, Michael Chertkov et al.

In this article, we present a method to reconstruct the topology of a partially observed radial network of linear dynamical systems with bi-directional interactions. Our approach exploits the structure of the inverse power spectral density matrix and recovers edges involving nodes up to four hops away in the underlying topology. We then present an algorithm with provable guarantees, which eliminates the spurious links obtained and also identifies the location of the unobserved nodes in the inferred topology. The algorithm recovers the exact topology of the network by using only time-series of the states at the observed nodes. The effectiveness of the method developed is demonstrated by applying it on a typical distribution system of the electric grid.

Abhay Singh Bhadoriya, Deepjyoti Deka, Kaarthik Sundar

The Multiple Traveling Salesman Problem (MTSP) extends the traveling salesman problem by assigning multiple salesmen to visit a set of targets from a common depot, with each target visited exactly once while minimizing total tour length. A common variant, the min-max MTSP, focuses on workload balance by minimizing the longest tour, but it is difficult to solve optimally due to weak linear relaxation bounds. This paper introduces two new parametric fairness-driven variants of the MTSP: the $\varepsilon$-Fair-MTSP and the $Δ$-Fair-MTSP, which promote equitable distribution of tour lengths while controlling overall cost. The $\varepsilon$-Fair-MTSP is formulated as a mixed-integer second-order cone program, while the $Δ$-Fair-MTSP is modeled as a mixed-integer linear program. We develop algorithms that guarantee global optimality for both formulations. Computational experiments on benchmark instances and real-world applications, including electric vehicle fleet routing, demonstrate their effectiveness. Furthermore, we show that the algorithms presented for the fairness-constrained MTSP variants can be used to obtain the Pareto front of a bi-objective optimization problem in which one objective minimizes the total tour length and the other balances the lengths of the individual tours. Overall, these fairness-constrained MTSP variants provide a practical and flexible alternative to the min-max MTSP.

Ali Jalilian, Deepjyoti Deka, Md. Umar Hashmi et al.

Dynamic operating envelopes (DOEs) provide a systematic framework to integrate the flexibility of distribution grid resources while safeguarding network limits such as line ratings and voltage bounds. However, the flexibility derived from individual DOEs is often restricted and conservative, especially when some resources can coordinate via communication with an aggregator. This paper presents a convex, geometry-aware framework for constructing DOE for distribution grid customers under partial coordination, with coordinated customers modeled through polytopal flexibility sets and non-coordinated customers through hyperrectangles. The framework additionally incorporates fairness constraints for export and import headroom allocated to the customers within the DOE design. To account for forecast uncertainty in inelastic injections, the DOE design is extended to a robust formulation for bounded uncertainty sets. Case studies on the European Low Voltage Test Feeder indicate that the proposed DOE construction expands total harnessed flexibility, while being consistent with network limits, export/import fairness constraints and is robust to forecast uncertainty. Specifically, coordinating 30% of customers increased the achievable aggregate active-power injection range by approximately 25% relative to the non-coordinated baseline.

Harish Doddi, Saurav Talukdar, Deepjyoti Deka et al.

Learning the structure of a network from time series data, in particular cyclostationary data, is of significant interest in many disciplines such as power grids, biology and finance. In this article, an algorithm is presented for reconstruction of the topology of a network of cyclostationary processes. To the best of our knowledge, this is the first work to guarantee exact recovery without any assumptions on the underlying structure. The method is based on a lifting technique by which cyclostationary processes are mapped to vector wide sense stationary processes and further on semi-definite properties of matrix Wiener filters for the said processes.We demonstrate the performance of the proposed algorithm on a Resistor-Capacitor network and present the accuracy of reconstruction for varying sample sizes.

Mishfad Shaikh Veedu, Deepjyoti Deka, Murti V. Salapaka

In this article, the optimal sample complexity of learning the underlying interactions or dependencies of a Linear Dynamical System (LDS) over a Directed Acyclic Graph (DAG) is studied. We call such a DAG underlying an LDS as dynamical DAG (DDAG). In particular, we consider a DDAG where the nodal dynamics are driven by unobserved exogenous noise sources that are wide-sense stationary (WSS) in time but are mutually uncorrelated, and have the same {power spectral density (PSD)}. Inspired by the static DAG setting, a metric and an algorithm based on the PSD matrix of the observed time series are proposed to reconstruct the DDAG. It is shown that the optimal sample complexity (or length of state trajectory) needed to learn the DDAG is $n=Θ(q\log(p/q))$, where $p$ is the number of nodes and $q$ is the maximum number of parents per node. To prove the sample complexity upper bound, a concentration bound for the PSD estimation is derived, under two different sampling strategies. A matching min-max lower bound using generalized Fano's inequality also is provided, thus showing the order optimality of the proposed algorithm.

Aron Brenner, Deepjyoti Deka, Line Roald et al.

Large, spatially flexible electricity consumers such as data centers can reallocate demand across locations, influencing dispatch and prices in wholesale electricity markets. While flexible load is often assumed to improve system efficiency, this intuition typically relies on price-taking behavior. We study price-anticipatory spatial load shifting by modeling a large flexible consumer as a Stackelberg leader interacting with DC optimal power flow (DC-OPF) based market clearing. We show that decentralized, cost-minimizing load shifting need not align with system operating cost minimization, and that misalignment arises at boundaries between DC-OPF operating regimes, where small changes in load can induce discrete changes in marginal generators or congestion patterns. We evaluate strategic load shifting on the 73-bus RTS-GMLC test system, where findings indicate reductions in system operating cost in most hours, but misalignment in a subset of cases that are driven by redispatch at merit-order discontinuities. We find that these outcomes are primarily redistributive relative to a price-taking benchmark, reducing generator profits while lowering electricity procurement costs for both flexible and inflexible consumers, even in cases where total system operating costs increase.

Dirk Lauinger, Deepjyoti Deka, Sungho Shin

Electricity distribution companies deploy battery storage to defer grid upgrades by reducing peak demand. In deregulated jurisdictions, such storage often sits idle because regulatory constraints bar participation in electricity markets. Here, we develop an optimization framework that, to our knowledge, provides the first formal model of market participation constraints within storage investment and operation planning. Applying the framework to a Massachusetts case study, we find that market participation delivers similar savings as peak demand reduction. Under current conditions, market participation does not increase storage investment, but at very low storage costs, could incentivize deployment beyond local distribution needs. This might run contrary to the separation of distribution from generation in deregulated markets. Our framework can mitigate this concern by identifying investment levels appropriate for local distribution needs.

Parikshit Pareek, Deepjyoti Deka, Sidhant Misra

This work presents an efficient data-driven method to construct probabilistic voltage envelopes (PVE) using power flow learning in grids with network contingencies. First, a network-aware Gaussian process (GP) termed Vertex-Degree Kernel (VDK-GP), developed in prior work, is used to estimate voltage-power functions for a few network configurations. The paper introduces a novel multi-task vertex degree kernel (MT-VDK) that amalgamates the learned VDK-GPs to determine power flows for unseen networks, with a significant reduction in the computational complexity and hyperparameter requirements compared to alternate approaches. Simulations on the IEEE 30-Bus network demonstrate the retention and transfer of power flow knowledge in both N-1 and N-2 contingency scenarios. The MT-VDK-GP approach achieves over 50% reduction in mean prediction error for novel N-1 contingency network configurations in low training data regimes (50-250 samples) over VDK-GP. Additionally, MT-VDK-GP outperforms a hyper-parameter based transfer learning approach in over 75% of N-2 contingency network structures, even without historical N-2 outage data. The proposed method demonstrates the ability to achieve PVEs using sixteen times fewer power flow solutions compared to Monte-Carlo sampling-based methods.

Parikshit Pareek, Sidhant Misra, Deepjyoti Deka

The absence of formal performance guarantees in machine learning (ML) has limited its adoption for safety-critical power system applications, where confidence and interpretability are as vital as accuracy. In this work, we present a probabilistic guarantee for power flow learning and voltage risk estimation, derived through the framework of Gaussian Process (GP) regression. Specifically, we establish a bound on the expected estimation error that connects the GP's predictive variance to confidence in voltage risk estimates, ensuring statistical equivalence with Monte Carlo-based ACPF risk quantification. To enhance model learnability in the low-data regime, we first design the Vertex-Degree Kernel (VDK), a topology-aware additive kernel that decomposes voltage-load interactions into local neighborhoods for efficient large-scale learning. Building on this, we introduce a network-swipe active learning (AL) algorithm that adaptively samples informative operating points and provides a principled stopping criterion without requiring out-of-sample validation. Together, these developments mitigate the principal bottleneck of ML-based power flow-its lack of guaranteed reliability-by combining data efficiency with analytical assurance. Empirical evaluations across IEEE 118-, 500-, and 1354-bus systems confirm that the proposed VDK-GP achieves mean absolute voltage errors below 1E-03 p.u., reproduces Monte Carlo-level voltage risk estimates with 15x fewer ACPF computations, and achieves over 120x reduction in evaluation time while conservatively bounding violation probabilities.

Mile Mitrovic, Aleksandr Lukashevich, Petr Vorobev et al.

The alternating current (AC) chance-constrained optimal power flow (CC-OPF) problem addresses the economic efficiency of electricity generation and delivery under generation uncertainty. The latter is intrinsic to modern power grids because of the high amount of renewables. Despite its academic success, the AC CC-OPF problem is highly nonlinear and computationally demanding, which limits its practical impact. For improving the AC-OPF problem complexity/accuracy trade-off, the paper proposes a fast data-driven setup that uses the sparse and hybrid Gaussian processes (GP) framework to model the power flow equations with input uncertainty. We advocate the efficiency of the proposed approach by a numerical study over multiple IEEE test cases showing up to two times faster and more accurate solutions compared to the state-of-the-art methods.

Shaohui Liu, Sungho Shin, Deepjyoti Deka

Enabling continued data-center growth under increasing grid stress motivates closer coordination between flexible computing demand and co-located battery energy storage systems (BESS) to improve site operations and provide grid services. This paper develops a robust co-optimization framework for day-ahead operation of data centers with co-located BESS under utility-imposed interconnection limits on peak load and ramping. The model jointly considers deadline-constrained computing workloads, managed through workload scheduling and dynamic voltage and frequency scaling (DVFS), together with degradation-aware BESS dispatch to enable cost optimization and participation in ancillary-service markets. Case studies based on real-world market and workload data show that the proposed framework yields feasible day-ahead schedules across a range of operating conditions, with substantially larger benefits when interconnection constraints become binding. Under baseline conditions, BESS value is derived from both ancillary-service participation and improved workload and energy management. Under stressed peak-load and ramping limits, however, the daily value of BESS increases by a factor of two or more, driven primarily \revise{by BESS actions to reduce the potential incompletion in the schedulable workload while complying with interconnection constraints}. Under tight peak-load caps, workload composition also matters where a higher share of non-schedulable jobs can increase operating cost by more than 25\% relative to more flexible workload mixes. \revise{Additionally, DVFS studies further show that processor-level control is a material flexibility lever under tight load limits.} These results demonstrate that coordinated compute-storage flexibility can materially expand the operational headroom and grid value of data centers, especially under increasingly scarce grid capacity.

Nikolay Stulov, Dejan J Sobajic, Yury Maximov et al.

In this work we investigate approaches to reconstruct generator models from measurements available at the generator terminal bus using machine learning (ML) techniques. The goal is to develop an emulator which is trained online and is capable of fast predictive computations. The training is illustrated on synthetic data generated based on available open-source dynamical generator model. Two ML techniques were developed and tested: (a) standard vector auto-regressive (VAR) model; and (b) novel customized long short-term memory (LSTM) deep learning model. Trade-offs in reconstruction ability between computationally light but linear AR model and powerful but computationally demanding LSTM model are established and analyzed.

Anirudh Rayas, Jiajun Cheng, Rajasekhar Anguluri et al.

Complex networked systems driven by latent inputs are common in fields like neuroscience, finance, and engineering. A key inference problem here is to learn edge connectivity from node outputs (potentials). We focus on systems governed by steady-state linear conservation laws: $X_t = {L^{\ast}}Y_{t}$, where $X_t, Y_t \in \mathbb{R}^p$ denote inputs and potentials, respectively, and the sparsity pattern of the $p \times p$ Laplacian $L^{\ast}$ encodes the edge structure. Assuming $X_t$ to be a wide-sense stationary stochastic process with a known spectral density matrix, we learn the support of $L^{\ast}$ from temporally correlated samples of $Y_t$ via an $\ell_1$-regularized Whittle's maximum likelihood estimator (MLE). The regularization is particularly useful for learning large-scale networks in the high-dimensional setting where the network size $p$ significantly exceeds the number of samples $n$. We show that the MLE problem is strictly convex, admitting a unique solution. Under a novel mutual incoherence condition and certain sufficient conditions on $(n, p, d)$, we show that the ML estimate recovers the sparsity pattern of $L^\ast$ with high probability, where $d$ is the maximum degree of the graph underlying $L^{\ast}$. We provide recovery guarantees for $L^\ast$ in element-wise maximum, Frobenius, and operator norms. Finally, we complement our theoretical results with several simulation studies on synthetic and benchmark datasets, including engineered systems (power and water networks), and real-world datasets from neural systems (such as the human brain).

Daniel Glover, Parikshit Pareek, Deepjyoti Deka et al.

Learning-based approaches are increasingly leveraged to manage and coordinate the operation of grid-edge resources in active power distribution networks. Among these, model-based techniques stand out for their superior data efficiency and robustness compared to model-free methods. However, effective model learning requires a learning-based approximator for the underlying power flow model. This study extends existing work by introducing a data-driven power flow method based on Gaussian Processes (GPs) to approximate the multiphase power flow model, by mapping net load injections to nodal voltages. Simulation results using the IEEE 123-bus and 8500-node distribution test feeders demonstrate that the trained GP model can reliably predict the nonlinear power flow solutions with minimal training data. We also conduct a comparative analysis of the training efficiency and testing performance of the proposed GP-based power flow approximator against a deep neural network-based approximator, highlighting the advantages of our data-efficient approach. Results over realistic operating conditions show that despite an 85% reduction in the training sample size (corresponding to a 92.8% improvement in training time), GP models produce a 99.9% relative reduction in mean absolute error compared to the baselines of deep neural networks.

Sina Sontowski, Nigel Lawrence, Deepjyoti Deka et al.

As cyber-attacks against critical infrastructure become more frequent, it is increasingly important to be able to rapidly identify and respond to these threats. This work investigates two independent systems with overlapping electrical measurements with the goal to more rapidly identify anomalies. The independent systems include HIST, a SCADA historian, and ION, an automatic meter reading system (AMR). While prior research has explored the benefits of fusing measurements, the possibility of overlapping measurements from an existing electrical system has not been investigated. To that end, we explore the potential benefits of combining overlapping measurements both to improve the speed/accuracy of anomaly detection and to provide additional validation of the collected measurements. In this paper, we show that merging overlapping measurements provide a more holistic picture of the observed systems. By applying Dynamic Time Warping more anomalies were found -- specifically, an average of 349 times more anomalies, when considering anomalies from both overlapping measurements. When merging the overlapping measurements, a percent change of anomalies of up to 785\% can be achieved compared to a non-merge of the data as reflected by experimental results.

Sarthak Gupta, Sidhant Misra, Deepjyoti Deka et al.

A prominent challenge to the safe and optimal operation of the modern power grid arises due to growing uncertainties in loads and renewables. Stochastic optimal power flow (SOPF) formulations provide a mechanism to handle these uncertainties by computing dispatch decisions and control policies that maintain feasibility under uncertainty. Most SOPF formulations consider simple control policies such as affine policies that are mathematically simple and resemble many policies used in current practice. Motivated by the efficacy of machine learning (ML) algorithms and the potential benefits of general control policies for cost and constraint enforcement, we put forth a deep neural network (DNN)-based policy that predicts the generator dispatch decisions in real time in response to uncertainty. The weights of the DNN are learnt using stochastic primal-dual updates that solve the SOPF without the need for prior generation of training labels and can explicitly account for the feasibility constraints in the SOPF. The advantages of the DNN policy over simpler policies and their efficacy in enforcing safety limits and producing near optimal solutions are demonstrated in the context of a chance constrained formulation on a number of test cases.

Harish Doddi, Deepjyoti Deka, Saurav Talukdar et al.

We consider a networked linear dynamical system with $p$ agents/nodes. We study the problem of learning the underlying graph of interactions/dependencies from observations of the nodal trajectories over a time-interval $T$. We present a regularized non-casual consistent estimator for this problem and analyze its sample complexity over two regimes: (a) where the interval $T$ consists of $n$ i.i.d. observation windows of length $T/n$ (restart and record), and (b) where $T$ is one continuous observation window (consecutive). Using the theory of $M$-estimators, we show that the estimator recovers the underlying interactions, in either regime, in a time-interval that is logarithmic in the system size $p$. To the best of our knowledge, this is the first work to analyze the sample complexity of learning linear dynamical systems \emph{driven by unobserved not-white wide-sense stationary (WSS) inputs}.

Wenting Li, Deepjyoti Deka

Electrical faults may trigger blackouts or wildfires without timely monitoring and control strategy. Traditional solutions for locating faults in distribution systems are not real-time when network observability is low, while novel black-box machine learning methods are vulnerable to stochastic environments. We propose a novel Physics-Preserved Graph Network (PPGN) architecture to accurately locate faults at the node level with limited observability and labeled training data. PPGN has a unique two-stage graph neural network architecture. The first stage learns the graph embedding to represent the entire network using a few measured nodes. The second stage finds relations between the labeled and unlabeled data samples to further improve the location accuracy. We explain the benefits of the two-stage graph configuration through a random walk equivalence. We numerically validate the proposed method in the IEEE 123-node and 37-node test feeders, demonstrating the superior performance over three baseline classifiers when labeled training data is limited, and loads and topology are allowed to vary.

Harish Doddi, Deepjyoti Deka, Murti Salapaka

Topology learning of networked dynamical systems is an important problem with implications to optimal control, decision-making over networks, cybersecurity and safety. The majority of prior work in consistent topology estimation relies on dynamical systems excited by temporally uncorrelated processes. In this article, we present a novel algorithm for guaranteed topology learning of networks that are excited by temporally (colored) cyclostationary processes, which encompasses a wide range of temporal correlation including wide-sense stationarity. Furthermore, unlike prior work, the framework applies to linear dynamic system with complex valued dependencies, and leverages group lasso regularization for effective learning of the network structure. In the second part of the article, we analyze conditions for consistent topology learning for bidirected tree networks when a subset of the network is unobserved. Here, the full topology along with unobserved nodes are recovered from observed node's time-series alone. Our theoretical contributions are validated on simulated data as well as on real-world climate data.

Ali Hassan, Samrat Acharya, Michael Chertkov et al.

Due to proliferation of energy efficiency measures and availability of the renewable energy resources, traditional energy infrastructure systems (electricity, heat, gas) can no longer be operated in a centralized manner under the assumption that consumer behavior is inflexible, i.e. cannot be adjusted in return for an adequate incentive. To allow for a less centralized operating paradigm, consumer-end perspective and abilities should be integrated in current dispatch practices and accounted for in switching between different energy sources not only at the system but also at the individual consumer level. Since consumers are confined within different built environments, this paper looks into an opportunity to control energy consumption of an aggregation of many residential, commercial and industrial consumers, into an ensemble. This ensemble control becomes a modern demand response contributor to the set of modeling tools for multi-energy infrastructure systems.

Deepjyoti Deka, Harish Doddi, Sidhant Misra et al.

Estimating the structure of physical flow networks such as power grids is critical to secure delivery of energy. This paper discusses statistical structure estimation in power grids in the "under-excited" regime, where a subset of internal nodes do not have external injection. Prior estimation algorithms based on nodal potentials or voltages fail in the under-excited regime. We propose a novel topology learning algorithm for learning underexcited general (non-radial) networks based on physics-informed conservation laws. We prove the asymptotic correctness of our algorithm for grids with non-adjacent under-excited internal nodes. More importantly, we theoretically analyze our algorithm's efficacy under noisy measurements, and determine bounds on maximum noise under which asymptotically correct recovery is guaranteed. Our approach is validated through simulations with non-linear voltage samples generated on test grids with real injection data

Ali Hassan, Deepjyoti Deka, Michael Chertkov et al.

Demand response (DR) programs aim to engage distributed small-scale flexible loads, such as thermostatically controllable loads (TCLs), to provide various grid support services. Linearly Solvable Markov Decision Process (LS-MDP), a variant of the traditional MDP, is used to model aggregated TCLs. Then, a model-free reinforcement learning technique called Z-learning is applied to learn the value function and derive the optimal policy for the DR aggregator to control TCLs. The learning process is robust against uncertainty that arises from estimating the passive dynamics of the aggregated TCLs. The efficiency of this data-driven learning is demonstrated through simulations on Heating, Cooling & Ventilation (HVAC) units in a testbed neighborhood of residential houses.

Christopher Hannon, Deepjyoti Deka, Dong Jin et al.

Ensuring secure and reliable operations of the power grid is a primary concern of system operators. Phasor measurement units (PMUs) are rapidly being deployed in the grid to provide fast-sampled operational data that should enable quicker decision-making. This work presents a general interpretable framework for analyzing real-time PMU data, and thus enabling grid operators to understand the current state and to identify anomalies on the fly. Applying statistical learning tools on the streaming data, we first learn an effective dynamical model to describe the current behavior of the system. Next, we use the probabilistic predictions of our learned model to define in a principled way an efficient anomaly detection tool. Finally, the last module of our framework produces on-the-fly classification of the detected anomalies into common occurrence classes using features that grid operators are familiar with. We demonstrate the efficacy of our interpretable approach through extensive numerical experiments on real PMU data collected from a transmission operator in the USA.

Wenting Li, Deepjyoti Deka, Michael Chertkov et al.

Diverse fault types, fast re-closures, and complicated transient states after a fault event make real-time fault location in power grids challenging. Existing localization techniques in this area rely on simplistic assumptions, such as static loads, or require much higher sampling rates or total measurement availability. This paper proposes a faulted line localization method based on a Convolutional Neural Network (CNN) classifier using bus voltages. Unlike prior data-driven methods, the proposed classifier is based on features with physical interpretations that improve the robustness of the location performance. The accuracy of our CNN based localization tool is demonstrably superior to other machine learning classifiers in the literature. To further improve the location performance, a joint phasor measurement units (PMU) placement strategy is proposed and validated against other methods. A significant aspect of our methodology is that under very low observability (7% of buses), the algorithm is still able to localize the faulted line to a small neighborhood with high probability. The performance of our scheme is validated through simulations of faults of various types in the IEEE 39-bus and 68-bus power systems under varying uncertain conditions, system observability, and measurement quality.

Saurav Talukdar, Deepjyoti Deka, Harish Doddi et al.

Learning influence pathways of a network of dynamically related processes from observations is of considerable importance in many disciplines. In this article, influence networks of agents which interact dynamically via linear dependencies are considered. An algorithm for the reconstruction of the topology of interaction based on multivariate Wiener filtering is analyzed. It is shown that for a vast and important class of interactions, that respect flow conservation, the topology of the interactions can be exactly recovered. The class of problems where reconstruction is guaranteed to be exact includes power distribution networks, dynamic thermal networks and consensus networks. The efficacy of the approach is illustrated through simulation and experiments on consensus networks, IEEE power distribution networks and thermal dynamics of buildings.

Harish Doddi, Saurav Talukdar, Deepjyoti Deka et al.

System identification of smart buildings is necessary for their optimal control and application in demand response. The thermal response of a building around an operating point can be modeled using a network of interconnected resistors with capacitors at each node/zone called RC network. The development of the RC network involves two phases: obtaining the network topology, and estimating thermal resistances and capacitance's. In this article, we present a provable method to reconstruct the interaction topology of thermal zones of a building solely from temperature measurements. We demonstrate that our learning algorithm accurately reconstructs the interaction topology for a $5$ zone office building in EnergyPlus with real-world conditions. We show that our learning algorithm is able to recover the network structure in scenarios where prior research prove insufficient.

Colin Grudzien, Deepjyoti Deka, Michael Chertkov et al.

The large size of multiscale, distribution and transmission, power grids hinder fast system-wide estimation and real-time control and optimization of operations. This paper studies graph reduction methods of power grids that are favorable for fast simulations and follow-up applications. While the classical Kron reduction has been successful in reduced order modeling of power grids with traditional, hierarchical design, the selection of reference nodes for the reduced model in a multiscale, distribution and transmission, network becomes ambiguous. In this work we extend the use of the iterative Kron reduction by utilizing the electric grid's graph topology for the selection of reference nodes, consistent with the design features of multiscale networks. Additionally, we propose further reductions by aggregation of coherent subnetworks of triangular meshes, based on the graph topology and network characteristics, in order to preserve currents and build another power-flow equivalent network. Our reductions are achieved through the use of iterative aggregation of sub-graphs that include general tree structures, lines, and triangles. Important features of our reduction algorithms include that: (i) the reductions are, either, equivalent to the Kron reduction, or otherwise produce a power-flow equivalent network; (ii) due to the former mentioned power-flow equivalence, the reduced network can model the dynamic of the swing equations for a lossless, inductive, steady state network; (iii) the algorithms efficiently utilize hash-tables to store the sequential reduction steps.

Deepjyoti Deka, Michael Chertkov, Scott Backhaus

Distribution grid is the medium and low voltage part of a large power system. Structurally, the majority of distribution networks operate radially, such that energized lines form a collection of trees, i.e. forest, with a substation being at the root of any tree. The operational topology/forest may change from time to time, however tracking these changes, even though important for the distribution grid operation and control, is hindered by limited real-time monitoring. This paper develops a learning framework to reconstruct radial operational structure of the distribution grid from synchronized voltage measurements in the grid subject to the exogenous fluctuations in nodal power consumption. To detect operational lines our learning algorithm uses conditional independence tests for continuous random variables that is applicable to a wide class of probability distributions of the nodal consumption and Gaussian injections in particular. Moreover, our algorithm applies to the practical case of unbalanced three-phase power flow. Algorithm performance is validated on AC power flow simulations over IEEE distribution grid test cases.

Andrey Y. Lokhov, Marc Vuffray, Dmitry Shemetov et al.

We consider the problem of reconstructing the dynamic state matrix of transmission power grids from time-stamped PMU measurements in the regime of ambient fluctuations. Using a maximum likelihood based approach, we construct a family of convex estimators that adapt to the structure of the problem depending on the available prior information. The proposed method is fully data-driven and does not assume any knowledge of system parameters. It can be implemented in near real-time and requires a small amount of data. Our learning algorithms can be used for model validation and calibration, and can also be applied to related problems of system stability, detection of forced oscillations, generation re-dispatch, as well as to the estimation of the system state.

Saurav Talukdar, Deepjyoti Deka, Sandeep Attree et al.

In this article, we present a method to learn the interaction topology of a network of agents undergoing linear consensus updates in a non invasive manner. Our approach is based on multivariate Wiener filtering, which is known to recover spurious edges apart from the true edges in the topology. The main contribution of this work is to show that in the case of undirected consensus networks, all spurious links obtained using Wiener filtering can be identified using frequency response of the Wiener filters. Thus, the exact interaction topology of the agents is unveiled. The method presented requires time series measurements of the state of the agents and does not require any knowledge of link weights. To the best of our knowledge this is the first approach that provably reconstructs the structure of undirected consensus networks with correlated noise. We illustrate the effectiveness of the method developed through numerical simulations as well as experiments on a five node network of Raspberry Pis.

Deepjyoti Deka, Saurav Talukdar, Michael Chertkov et al.

The topology of a power grid affects its dynamic operation and settlement in the electricity market. Real-time topology identification can enable faster control action following an emergency scenario like failure of a line. This article discusses a graphical model framework for topology estimation in bulk power grids (both loopy transmission and radial distribution) using measurements of voltage collected from the grid nodes. The graphical model for the probability distribution of nodal voltages in linear power flow models is shown to include additional edges along with the operational edges in the true grid. Our proposed estimation algorithms first learn the graphical model and subsequently extract the operational edges using either thresholding or a neighborhood counting scheme. For grid topologies containing no three-node cycles (two buses do not share a common neighbor), we prove that an exact extraction of the operational topology is theoretically guaranteed. This includes a majority of distribution grids that have radial topologies. For grids that include cycles of length three, we provide sufficient conditions that ensure existence of algorithms for exact reconstruction. In particular, for grids with constant impedance per unit length and uniform injection covariances, this observation leads to conditions on geographical placement of the buses. The performance of algorithms is demonstrated in test case simulations.

Saurav Talukdar, Deepjyoti Deka, Donatello Materassi et al.

In this article we present a method to reconstruct the interconnectedness of dynamically related stochastic processes, where the interactions are bi-directional and the underlying topology is a tree. Our approach is based on multivariate Wiener filtering which recovers spurious edges apart from the true edges in the topology reconstruction. The main contribution of this work is to show that all spurious links obtained using Wiener filtering can be eliminated if the underlying topology is a tree based on which we present a three stage network reconstruction procedure for trees. We illustrate the effectiveness of the method developed by applying it on a typical distribution system of the electric grid.