Geert Deconinck

Jonas Engels, Bert Claessens, Geert Deconinck

Optimally combining frequency control with self-consumption can increase revenues from battery storage systems installed behind-the-meter. This work presents an optimized control strategy that allows a battery to be used simultaneously for self-consumption and primary frequency control. Therein, it addresses two stochastic problems: the delivery of primary frequency control with a battery and the use of the battery for self-consumption. We propose a linear recharging policy to regulate the state of charge of the battery while providing primary frequency control. Formulating this as a chance-constrained problem, we can ensure that the risk of battery constraint violations stays below a predefined probability. We use robust optimization as a safe approximation to the chance-constraints, which allows to make the risk of constraint violation arbitrarily low, while keeping the problem tractable and offering maximum reserve capacity. Simulations with real frequency measurements prove the effectiveness of the designed recharging strategy. We adopt a rule-based policy for self-consumption, which is optimized using stochastic programming. This policy allows to reserve more energy and power of the battery on moments when expected consumption or production is higher, while using other moments for recharging from primary frequency control. We show that optimally combining the two services increases value from batteries significantly.

Jonas Engels, Bert Claessens, Geert Deconinck

Optimal investment in battery energy storage systems, taking into account degradation, sizing and control, is crucial for the deployment of battery storage, of which providing frequency control is one of the major applications. In this paper, we present a holistic, data-driven framework to determine the optimal investment, size and controller of a battery storage system providing frequency control. We optimised the controller towards minimum degradation and electricity costs over its lifetime, while ensuring the delivery of frequency control services compliant with regulatory requirements. We adopted a detailed battery model, considering the dynamics and degradation when exposed to actual frequency data. Further, we used a stochastic optimisation objective while constraining the probability on unavailability to deliver the frequency control service. Through a thorough analysis, we were able to decrease the amount of data needed and thereby decrease the execution time while keeping the approximation error within limits. Using the proposed framework, we performed a techno-economic analysis of a battery providing 1 MW capacity in the German primary frequency control market. Results showed that a battery rated at 1.6 MW, 1.6 MWh has the highest net present value, yet this configuration is only profitable if costs are low enough or in case future frequency control prices do not decline too much. It transpires that calendar ageing drives battery degradation, whereas cycle ageing has less impact.

Jonas Engels, Bert Claessens, Geert Deconinck

Providing frequency control services with flexible assets connected to the low-voltage distribution grid, amongst which residential battery storage or electrical hot water boilers, can lead to congestion problems and voltage issues in the distribution grid. In order to mitigate these problems, a new regulation has been put in place in Belgium, imposing a specific constraint: in any circle with a radius of 100m, there can be at maximum 10 connection points providing frequency control at any time. This paper presents an impact analysis and a coordination strategy of a Flexibility Service Provider (FSP) that operates a pool of assets and is exposed to this new regulatory constraint. Results show that at 5% participation, only 90% of total control capacity can be used, with a large difference between neighbourhoods with different population densities. A distributed optimization framework to coordinate the assets arises naturally, in which the assets are able to keep their local cost functions private and only have to communicate with neighbouring assets that are geographically close, and with the FSP. Analysis of the proposed distributed optimization algorithm shows a clear trade-off between optimality gap, owing to the mixed-integer nature of the problem, and iterations to convergence.

Thijs Peirelinck, Chris Hermans, Fred Spiessens et al.

Residential demand response programs aim to activate demand flexibility at the household level. In recent years, reinforcement learning (RL) has gained significant attention for these type of applications. A major challenge of RL algorithms is data efficiency. New RL algorithms, such as proximal policy optimisation (PPO), have tried to increase data efficiency. Additionally, combining RL with transfer learning has been proposed in an effort to mitigate this challenge. In this work, we further improve upon state-of-the-art transfer learning performance by incorporating demand response domain knowledge into the learning pipeline. We evaluate our approach on a demand response use case where peak shaving and self-consumption is incentivised by means of a capacity tariff. We show our adapted version of PPO, combined with transfer learning, reduces cost by 14.51% compared to a regular hysteresis controller and by 6.68% compared to traditional PPO.

Tohid Behdadnia, Geert Deconinck

In modern highly interconnected power grids, automatic generation control (AGC) is crucial in maintaining the stability of the power grid. The dependence of the AGC system on the information and communications technology (ICT) system makes it vulnerable to various types of cyber-attacks. Thus, information flow (IF) analysis and anomaly detection became paramount for preventing cyber attackers from driving the cyber-physical power system (CPPS) to instability. In this paper, the ICT network traffic rules in CPPSs are explored and the frequency domain features of the ICT network traffic are extracted, basically for developing a robust learning algorithm that can learn the normal traffic pattern based on the ResNeSt convolutional neural network (CNN). Furthermore, to overcome the problem of insufficient abnormal traffic labeled samples, transfer learning approach is used. In the proposed data-driven-based method the deep learning model is trained by traffic frequency features, which makes our model robust against AGC's parameters uncertainties and modeling nonlinearities.

Yi Dong, Yingjie Wang, Mariana Gama et al.

In the realm of power systems, the increasing involvement of residential users in load forecasting applications has heightened concerns about data privacy. Specifically, the load data can inadvertently reveal the daily routines of residential users, thereby posing a risk to their property security. While federated learning (FL) has been employed to safeguard user privacy by enabling model training without the exchange of raw data, these FL models have shown vulnerabilities to emerging attack techniques, such as Deep Leakage from Gradients and poisoning attacks. To counteract these, we initially employ a Secure-Aggregation (SecAgg) algorithm that leverages multiparty computation cryptographic techniques to mitigate the risk of gradient leakage. However, the introduction of SecAgg necessitates the deployment of additional sub-center servers for executing the multiparty computation protocol, thereby escalating computational complexity and reducing system robustness, especially in scenarios where one or more sub-centers are unavailable. To address these challenges, we introduce a Markovian Switching-based distributed training framework, the convergence of which is substantiated through rigorous theoretical analysis. The Distributed Markovian Switching (DMS) topology shows strong robustness towards the poisoning attacks as well. Case studies employing real-world power system load data validate the efficacy of our proposed algorithm. It not only significantly minimizes communication complexity but also maintains accuracy levels comparable to traditional FL methods, thereby enhancing the scalability of our load forecasting algorithm.

Glenn Ceusters, Román Cantú Rodríguez, Alberte Bouso García et al.

Model-predictive-control (MPC) offers an optimal control technique to establish and ensure that the total operation cost of multi-energy systems remains at a minimum while fulfilling all system constraints. However, this method presumes an adequate model of the underlying system dynamics, which is prone to modelling errors and is not necessarily adaptive. This has an associated initial and ongoing project-specific engineering cost. In this paper, we present an on- and off-policy multi-objective reinforcement learning (RL) approach, that does not assume a model a priori, benchmarking this against a linear MPC (LMPC - to reflect current practice, though non-linear MPC performs better) - both derived from the general optimal control problem, highlighting their differences and similarities. In a simple multi-energy system (MES) configuration case study, we show that a twin delayed deep deterministic policy gradient (TD3) RL agent offers potential to match and outperform the perfect foresight LMPC benchmark (101.5%). This while the realistic LMPC, i.e. imperfect predictions, only achieves 98%. While in a more complex MES system configuration, the RL agent's performance is generally lower (94.6%), yet still better than the realistic LMPC (88.9%). In both case studies, the RL agents outperformed the realistic LMPC after a training period of 2 years using quarterly interactions with the environment. We conclude that reinforcement learning is a viable optimal control technique for multi-energy systems given adequate constraint handling and pre-training, to avoid unsafe interactions and long training periods, as is proposed in fundamental future work.

Frederik Ruelens, Bert J. Claessens, Peter Vrancx et al.

This paper considers a demand response agent that must find a near-optimal sequence of decisions based on sparse observations of its environment. Extracting a relevant set of features from these observations is a challenging task and may require substantial domain knowledge. One way to tackle this problem is to store sequences of past observations and actions in the state vector, making it high dimensional, and apply techniques from deep learning. This paper investigates the capabilities of different deep learning techniques, such as convolutional neural networks and recurrent neural networks, to extract relevant features for finding near-optimal policies for a residential heating system and electric water heater that are hindered by sparse observations. Our simulation results indicate that in this specific scenario, feeding sequences of time-series to an LSTM network, which is a specific type of recurrent neural network, achieved a higher performance than stacking these time-series in the input of a convolutional neural network or deep neural network.

Wim Pessemier, Gert Raskin, Hans Van Winckel et al.

Even though modern service-oriented and data-oriented architectures promise to deliver loosely coupled control systems, they are inherently brittle as they commonly depend on a priori agreed interfaces and data models. At the same time, the Semantic Web and a whole set of accompanying standards and tools are emerging, advocating ontologies as the basis for knowledge exchange. In this paper we aim to identify a number of key ideas from the myriad of knowledge-based practices that can readily be implemented by control systems today. We demonstrate with a practical example (a three-channel imager for the Mercator Telescope) how ontologies developed in the Web Ontology Language (OWL) can serve as a meta-model for our instrument, covering as many engineering aspects of the project as needed. We show how a concrete system model can be built on top of this meta-model via a set of Domain Specific Languages (DSLs), supporting both formal verification and the generation of software and documentation artifacts. Finally we reason how the available semantics can be exposed at run-time by adding a "semantic layer" that can be browsed, queried, monitored etc. by any OPC UA-enabled client.