Dirk Witthaut

Jonas Hörsch, Henrik Ronellenfitsch, Dirk Witthaut et al.

Linear optimal power flow (LOPF) algorithms use a linearization of the alternating current (AC) load flow equations to optimize generator dispatch in a network subject to the loading constraints of the network branches. Common algorithms use the voltage angles at the buses as optimization variables, but alternatives can be computationally advantageous. In this article we provide a review of existing methods and describe a new formulation that expresses the loading constraints directly in terms of the flows themselves, using a decomposition of the network graph into a spanning tree and closed cycles. We provide a comprehensive study of the computational performance of the various formulations, in settings that include computationally challenging applications such as multi-period LOPF with storage dispatch and generation capacity expansion. We show that the new formulation of the LOPF solves up to 7 times faster than the angle formulation using a commercial linear programming solver, while another existing cycle-based formulation solves up to 20 times faster, with an average speed-up of factor 3 for the standard networks considered here. If generation capacities are also optimized, the average speed-up rises to a factor of 12, reaching up to factor 213 in a particular instance. The speed-up is largest for networks with many buses and decentral generators throughout the network, which is highly relevant given the rise of distributed renewable generation and the computational challenge of operation and planning in such networks.

Eike Cramer, Dirk Witthaut, Alexander Mitsos et al.

Electricity is traded on various markets with different time horizons and regulations. Short-term intraday trading becomes increasingly important due to the higher penetration of renewables. In Germany, the intraday electricity price typically fluctuates around the day-ahead price of the European Power EXchange (EPEX) spot markets in a distinct hourly pattern. This work proposes a probabilistic modeling approach that models the intraday price difference to the day-ahead contracts. The model captures the emerging hourly pattern by considering the four 15 min intervals in each day-ahead price interval as a four-dimensional joint probability distribution. The resulting nontrivial, multivariate price difference distribution is learned using a normalizing flow, i.e., a deep generative model that combines conditional multivariate density estimation and probabilistic regression. Furthermore, this work discusses the influence of different external impact factors based on literature insights and impact analysis using explainable artificial intelligence (XAI). The normalizing flow is compared to an informed selection of historical data and probabilistic forecasts using a Gaussian copula and a Gaussian regression model. Among the different models, the normalizing flow identifies the trends with the highest accuracy and has the narrowest prediction intervals. Both the XAI analysis and the empirical experiments highlight that the immediate history of the price difference realization and the increments of the day-ahead price have the most substantial impact on the price difference.

Henrik Ronellenfitsch, Debsankha Manik, Jonas Hörsch et al.

A new graph dual formalism is presented for the analysis of line outages in electricity networks. The dual formalism is based on a consideration of the flows around closed cycles in the network. After some exposition of the theory is presented, a new formula for the computation of Line Outage Distribution Factors (LODFs) is derived, which is not only computationally faster than existing methods, but also generalizes easily for multiple line outages and arbitrary changes to line series reactance. In addition, the dual formalism provides new physical insight for how the effects of line outages propagate through the network. For example, in a planar network a single line outage can be shown to induce monotonically decreasing flow changes, which are mathematically equivalent to an electrostatic dipole field.

Julius Trebbien, Leonardo Rydin Gorjão, Aaron Praktiknjo et al.

Electricity prices in liberalized markets are determined by the supply and demand for electric power, which are in turn driven by various external influences that vary strongly in time. In perfect competition, the merit order principle describes that dispatchable power plants enter the market in the order of their marginal costs to meet the residual load, i.e. the difference of load and renewable generation. Many market models implement this principle to predict electricity prices but typically require certain assumptions and simplifications. In this article, we present an explainable machine learning model for the prices on the German day-ahead market, which substantially outperforms a benchmark model based on the merit order principle. Our model is designed for the ex-post analysis of prices and thus builds on various external features. Using Shapley Additive exPlanation (SHAP) values, we can disentangle the role of the different features and quantify their importance from empiric data. Load, wind and solar generation are most important, as expected, but wind power appears to affect prices stronger than solar power does. Fuel prices also rank highly and show nontrivial dependencies, including strong interactions with other features revealed by a SHAP interaction analysis. Large generation ramps are correlated with high prices, again with strong feature interactions, due to the limited flexibility of nuclear and lignite plants. Our results further contribute to model development by providing quantitative insights directly from data.

Maurizio Titz, Sebastian Pütz, Dirk Witthaut

The transition to a sustainable energy supply challenges the operation of electric power systems in manifold ways. Transmission grid loads increase as wind and solar power are often installed far away from the consumers. In extreme cases, system operators must intervene via countertrading or redispatch to ensure grid stability. In this article, we provide a data-driven analysis of congestion in the German transmission grid. We develop an explainable machine learning model to predict the volume of redispatch and countertrade on an hourly basis. The model reveals factors that drive or mitigate grid congestion and quantifies their impact. We show that, as expected, wind power generation is the main driver, but hydropower and cross-border electricity trading also play an essential role. Solar power, on the other hand, has no mitigating effect. Our results suggest that a change to the market design would alleviate congestion.

Maurizio Titz, Franz Kaiser, Johannes Kruse et al.

A reliable supply with electric power is vital for our society. Transmission line failures are among the biggest threats for power grid stability as they may lead to a splitting of the grid into mutual asynchronous fragments. New conceptual methods are needed to assess system stability that complement existing simulation models. In this article we propose a combination of network science metrics and machine learning models to predict the risk of desynchronisation events. Network science provides metrics for essential properties of transmission lines such as their redundancy or centrality. Machine learning models perform inherent feature selection and thus reveal key factors that determine network robustness and vulnerability. As a case study, we train and test such models on simulated data from several synthetic test grids. We find that the integrated models are capable of predicting desynchronisation events after line failures with an average precision greater than $0.996$ when averaging over all data sets. Learning transfer between different data sets is generally possible, at a slight loss of prediction performance. Our results suggest that power grid desynchronisation is essentially governed by only a few network metrics that quantify the networks ability to reroute flow without creating exceedingly high static line loadings.

Hannes Hilger, Dirk Witthaut, Manuel Dahmen et al.

Trading on the day-ahead electricity markets requires accurate information about the realization of electricity prices and the uncertainty attached to the predictions. Deriving accurate forecasting models presents a difficult task due to the day-ahead price's non-stationarity resulting from changing market conditions, e.g., due to changes resulting from the energy crisis in 2021. We present a probabilistic forecasting approach for day-ahead electricity prices using the fully data-driven deep generative model called normalizing flow. Our modeling approach generates full-day scenarios of day-ahead electricity prices based on conditional features such as residual load forecasts. Furthermore, we propose extended feature sets of prior realizations and a periodic retraining scheme that allows the normalizing flow to adapt to the changing conditions of modern electricity markets. Our results highlight that the normalizing flow generates high-quality scenarios that reproduce the true price distribution and yield accurate forecasts. Additionally, our analysis highlights how our improvements towards adaptations in changing regimes allow the normalizing flow to adapt to changing market conditions and enable continued sampling of high-quality day-ahead price scenarios.

Farnaz Basiri, Jose Casadiego, Marc Timme et al.

We develop methods to efficiently reconstruct the topology and line parameters of a power grid from the measurement of nodal variables. We propose two compressed sensing algorithms that minimize the amount of necessary measurement resources by exploiting network sparsity, symmetry of connections and potential prior knowledge about the connectivity. The algorithms are reciprocal to established state estimation methods, where nodal variables are estimated from few measurements given the network structure. Hence, they enable an advanced grid monitoring where both state and structure of a grid are subject to uncertainties or missing information.

Sebastian Pütz, Benjamin Schäfer, Dirk Witthaut et al.

The energy transition introduces more volatile energy sources into the power grids. In this context, power transfer between different synchronous areas through High Voltage Direct Current (HVDC) links becomes increasingly important. Such links can balance volatile generation by enabling long-distance transport or by leveraging their fast control behavior. Here, we investigate the interaction of power imbalances - represented through the power grid frequency - and power flows on HVDC links between synchronous areas in Europe. We use explainable machine learning to identify key dependencies and disentangle the interaction of critical features. Our results show that market-based HVDC flows introduce deterministic frequency deviations, which however can be mitigated through strict ramping limits. Moreover, varying HVDC operation modes strongly affect the interaction with the grid. In particular, we show that load-frequency control via HVDC links can both have control-like or disturbance-like impacts on frequency stability.

Julius Trebbien, Sebastian Pütz, Benjamin Schäfer et al.

Accurate forecasts of electricity prices are crucial for the management of electric power systems and the development of smart applications. European electricity prices have risen substantially and became highly volatile after the Russian invasion of Ukraine, challenging established forecasting methods. Here, we present a Long Short-Term Memory (LSTM) model for the German-Luxembourg day-ahead electricity prices addressing these challenges. The recurrent structure of the LSTM allows the model to adapt to trends, while the joint prediction of both mean and standard deviation enables a probabilistic prediction. Using a physics-inspired approach - superstatistics - to derive an explanation for the statistics of prices, we show that the LSTM model faithfully reproduces both prices and their volatility.

Nico Westerbeck, Leonard Hilfrich, Dirk Witthaut

Transmission Topology Optimization has great potential to improve efficiency and flexibility of grid operations through non-costly switching actions, but previous approaches struggle with runtime performance and scalability. In this work, we present an optimization approach that leverages GPU acceleration to speed up computations. In a genetic algorithm setting, topologies are randomly mutated and evaluated in parallel for multiple optimization criteria. Combined with a fully GPU-native DC loadflow solver, there is no CPU-GPU data transfer required in the DC optimization loop. Using a variant of the illumination algorithm MapElites, we efficiently generate a set of diverse candidate solutions on the pareto front. Together with an importing and AC validation step, we present an end-to-end optimization solution that runs in under 15 minutes. The approach is currently under evaluation by operational planning operators in two European TSOs. We furthermore open-source our code at github.com/eliagroup/ToOp.

Maurizio Titz, Dirk Witthaut, Joost van Dijk et al.

Topology optimization is a promising approach for mitigating congestion and managing changing grid conditions, but it is computationally challenging and requires approximations. Conventional distribution factors like PTDFs and LODFs, based on DC power flow, fail to capture voltage variations, reactive power, and losses, thereby limiting their use in detailed optimization tasks such as busbar splitting. This paper introduces generalized distribution factors derived from a voltage-sensitive linearization of the full AC power flow equations. The proposed formulation accurately reflects reactive power flows, Ohmic losses, and voltage deviations while remaining computationally efficient. We derive and evaluate generalized PTDFs, LODFs, and topology modification factors using matrix identities. We discuss potential applications including voltage-aware N-1 security analysis and topology optimization with a focus on busbar splitting. Numerical experiments demonstrate close agreement with full AC solutions, significantly outperforming the traditional DC approximation.

Eike Cramer, Leonardo Rydin Gorjão, Alexander Mitsos et al.

The design and operation of modern energy systems are heavily influenced by time-dependent and uncertain parameters, e.g., renewable electricity generation, load-demand, and electricity prices. These are typically represented by a set of discrete realizations known as scenarios. A popular scenario generation approach uses deep generative models (DGM) that allow scenario generation without prior assumptions about the data distribution. However, the validation of generated scenarios is difficult, and a comprehensive discussion about appropriate validation methods is currently lacking. To start this discussion, we provide a critical assessment of the currently used validation methods in the energy scenario generation literature. In particular, we assess validation methods based on probability density, auto-correlation, and power spectral density. Furthermore, we propose using the multifractal detrended fluctuation analysis (MFDFA) as an additional validation method for non-trivial features like peaks, bursts, and plateaus. As representative examples, we train generative adversarial networks (GANs), Wasserstein GANs (WGANs), and variational autoencoders (VAEs) on two renewable power generation time series (photovoltaic and wind from Germany in 2013 to 2015) and an intra-day electricity price time series form the European Energy Exchange in 2017 to 2019. We apply the four validation methods to both the historical and the generated data and discuss the interpretation of validation results as well as common mistakes, pitfalls, and limitations of the validation methods. Our assessment shows that no single method sufficiently characterizes a scenario but ideally validation should include multiple methods and be interpreted carefully in the context of scenarios over short time periods.

Johannes Kruse, Benjamin Schäfer, Dirk Witthaut

The transition to a renewable energy system poses challenges for power grid operation and stability. Secondary control is key in restoring the power system to its reference following a disturbance. Underestimating the necessary control capacity may require emergency measures, such as load shedding. Hence, a solid understanding of the emerging risks and the driving factors of control is needed. In this contribution, we establish an explainable machine learning model for the activation of secondary control power in Germany. Training gradient boosted trees, we obtain an accurate description of control activation. Using SHapely Additive exPlanation (SHAP) values, we investigate the dependency between control activation and external features such as the generation mix, forecasting errors, and electricity market data. Thereby, our analysis reveals drivers that lead to high reserve requirements in the German power system. Our transparent approach, utilizing open data and making machine learning models interpretable, opens new scientific discovery avenues.

Johannes Kruse, Benjamin Schäfer, Dirk Witthaut

Deterministic frequency deviations (DFDs) critically affect power grid frequency quality and power system stability. A better understanding of these events is urgently needed as frequency deviations have been growing in the European grid in recent years. DFDs are partially explained by the rapid adjustment of power generation following the intervals of electricity trading, but this intuitive picture fails especially before and around noonday. In this article, we provide a detailed analysis of DFDs and their relation to external features using methods from explainable Artificial Intelligence. We establish a machine learning model that well describes the daily cycle of DFDs and elucidate key interdependencies using SHapley Additive exPlanations (SHAP). Thereby, we identify solar ramps as critical to explain patterns in the Rate of Change of Frequency (RoCoF).

Johannes Kruse, Benjamin Schäfer, Dirk Witthaut

The power grid frequency is the central observable in power system control, as it measures the balance of electrical supply and demand. A reliable frequency forecast can facilitate rapid control actions and may thus greatly improve power system stability. Here, we develop a weighted-nearest-neighbor (WNN) predictor to investigate how predictable the frequency trajectories are. Our forecasts for up to one hour are more precise than averaged daily profiles and could increase the efficiency of frequency control actions. Furthermore, we gain an increased understanding of the specific properties of different synchronous areas by interpreting the optimal prediction parameters (number of nearest neighbors, the prediction horizon, etc.) in terms of the physical system. Finally, prediction errors indicate the occurrence of exceptional external perturbations. Overall, we provide a diagnostics tool and an accurate predictor of the power grid frequency time series, allowing better understanding of the underlying dynamics.

Henrik Ronellenfitsch, Marc Timme, Dirk Witthaut

Power Transfer Distribution Factors (PTDFs) play a crucial role in power grid security analysis, planning, and redispatch. Fast calculation of the PTDFs is therefore of great importance. In this paper, we present a non-approximative dual method of computing PTDFs. It uses power flows along topological cycles of the network but still relies on simple matrix algebra. At the core, our method changes the size of the matrix that needs to be inverted to calculate the PTDFs from $N\times N$, where $N$ is the number of buses, to $(L-N+1)\times (L-N+1)$, where $L$ is the number of lines and $L-N+1$ is the number of independent cycles (closed loops) in the network while remaining mathematically fully equivalent. For power grids containing a relatively small number of cycles, the method can offer a speedup of numerical calculations.