Karanjit Kalsi

Soumya Kundu, Karanjit Kalsi, Scott Backhaus

With increasing availability of communication and control infrastructure at the distribution systems, it is expected that the distributed energy resources (DERs) will take an active part in future power systems operations. One of the main challenges associated with integration of DERs in grid planning and control is in estimating the available flexibility in a collection of (heterogeneous) DERs, each of which may have local constraints that vary over time. In this work, we present a geometric approach for approximating the flexibility of a DER in modulating its active and reactive power consumption. The proposed method is agnostic about the type and model of the DERs, thereby facilitating a plug-and-play approach, and allows scalable aggregation of the flexibility of a collection of (heterogeneous) DERs at the distributed system level. Simulation results are presented to demonstrate the performance of the proposed method.

Fan Lu, Joel Mathias, Sean Meyn et al.

Convex Q-learning is a recent approach to reinforcement learning, motivated by the possibility of a firmer theory for convergence, and the possibility of making use of greater a priori knowledge regarding policy or value function structure. This paper explores algorithm design in the continuous time domain, with finite-horizon optimal control objective. The main contributions are (i) Algorithm design is based on a new Q-ODE, which defines the model-free characterization of the Hamilton-Jacobi-Bellman equation. (ii) The Q-ODE motivates a new formulation of Convex Q-learning that avoids the approximations appearing in prior work. The Bellman error used in the algorithm is defined by filtered measurements, which is beneficial in the presence of measurement noise. (iii) A characterization of boundedness of the constraint region is obtained through a non-trivial extension of recent results from the discrete time setting. (iv) The theory is illustrated in application to resource allocation for distributed energy resources, for which the theory is ideally suited.