Zaiyue Yang

Wayes Tushar, Bo Chai, Chau Yuen et al.

This paper studies the solution of joint energy storage (ES) ownership sharing between multiple shared facility controllers (SFCs) and those dwelling in a residential community. The main objective is to enable the residential units (RUs) to decide on the fraction of their ES capacity that they want to share with the SFCs of the community in order to assist them storing electricity, e.g., for fulfilling the demand of various shared facilities. To this end, a modified auction-based mechanism is designed that captures the interaction between the SFCs and the RUs so as to determine the auction price and the allocation of ES shared by the RUs that governs the proposed joint ES ownership. The fraction of the capacity of the storage that each RU decides to put into the market to share with the SFCs and the auction price are determined by a noncooperative Stackelberg game formulated between the RUs and the auctioneer. It is shown that the proposed auction possesses the incentive compatibility and the individual rationality properties, which are leveraged via the unique Stackelberg equilibrium (SE) solution of the game. Numerical experiments are provided to confirm the effectiveness of the proposed scheme.

Wayes Tushar, Chau Yuen, Bo Chai et al.

Successful deployment of smart grids necessitates experimental validities of their state-of-the-art designs in two-way communications, real-time demand response and monitoring of consumers' energy usage behavior. The objective is to observe consumers' energy usage pattern and exploit this information to assist the grid in designing incentives, energy management mechanisms, and real-time demand response protocols; so as help the grid achieving lower costs and improve energy supply stability. Further, by feeding the observed information back to the consumers instantaneously, it is also possible to promote energy efficient behavior among the users. To this end, this paper performs a literature survey on smart grid testbeds around the world, and presents the main accomplishments towards realizing a smart grid testbed at the Singapore University of Technology and Design (SUTD). The testbed is able to monitor, analyze and evaluate smart grid communication network design and control mechanisms, and test the suitability of various communications networks for both residential and commercial buildings. The testbeds are deployed within the SUTD student dormitories and the main university campus to monitor and record end-user energy consumption in real-time, which will enable us to design incentives, control algorithms and real-time demand response schemes. The testbed also provides an effective channel to evaluate the needs on communication networks to support various smart grid applications. In addition, our initial results demonstrate that our testbed can provide an effective platform to identify energy wastage, and prompt the needs of a secure communications channel as the energy usage pattern can provide privacy related information on individual user.

Tobi Michael Alabi, Nathan P. Lawrence, Lin Lu et al.

The carbon-capturing process with the aid of CO2 removal technology (CDRT) has been recognised as an alternative and a prominent approach to deep decarbonisation. However, the main hindrance is the enormous energy demand and the economic implication of CDRT if not effectively managed. Hence, a novel deep reinforcement learning agent (DRL), integrated with an automated hyperparameter selection feature, is proposed in this study for the real-time scheduling of a multi-energy system coupled with CDRT. Post-carbon capture systems (PCCS) and direct-air capture systems (DACS) are considered CDRT. Various possible configurations are evaluated using real-time multi-energy data of a district in Arizona and CDRT parameters from manufacturers' catalogues and pilot project documentation. The simulation results validate that an optimised soft-actor critic (SAC) algorithm outperformed the TD3 algorithm due to its maximum entropy feature. We then trained four (4) SAC agents, equivalent to the number of considered case studies, using optimised hyperparameter values and deployed them in real time for evaluation. The results show that the proposed DRL agent can meet the prosumers' multi-energy demand and schedule the CDRT energy demand economically without specified constraints violation. Also, the proposed DRL agent outperformed rule-based scheduling by 23.65%. However, the configuration with PCCS and solid-sorbent DACS is considered the most suitable configuration with a high CO2 captured-released ratio of 38.54, low CO2 released indicator value of 2.53, and a 36.5% reduction in CDR cost due to waste heat utilisation and high absorption capacity of the selected sorbent. However, the adoption of CDRT is not economically viable at the current carbon price. Finally, we showed that CDRT would be attractive at a carbon price of 400-450USD/ton with the provision of tax incentives by the policymakers.

Jun Sun, Gang Wang, Georgios B. Giannakis et al.

Motivated by the emerging use of multi-agent reinforcement learning (MARL) in engineering applications such as networked robotics, swarming drones, and sensor networks, we investigate the policy evaluation problem in a fully decentralized setting, using temporal-difference (TD) learning with linear function approximation to handle large state spaces in practice. The goal of a group of agents is to collaboratively learn the value function of a given policy from locally private rewards observed in a shared environment, through exchanging local estimates with neighbors. Despite their simplicity and widespread use, our theoretical understanding of such decentralized TD learning algorithms remains limited. Existing results were obtained based on i.i.d. data samples, or by imposing an `additional' projection step to control the `gradient' bias incurred by the Markovian observations. In this paper, we provide a finite-sample analysis of the fully decentralized TD(0) learning under both i.i.d. as well as Markovian samples, and prove that all local estimates converge linearly to a small neighborhood of the optimum. The resultant error bounds are the first of its type---in the sense that they hold under the most practical assumptions ---which is made possible by means of a novel multi-step Lyapunov analysis.

Jun Sun, Tianyi Chen, Georgios B. Giannakis et al.

The present paper develops a novel aggregated gradient approach for distributed machine learning that adaptively compresses the gradient communication. The key idea is to first quantize the computed gradients, and then skip less informative quantized gradient communications by reusing outdated gradients. Quantizing and skipping result in `lazy' worker-server communications, which justifies the term Lazily Aggregated Quantized gradient that is henceforth abbreviated as LAQ. Our LAQ can provably attain the same linear convergence rate as the gradient descent in the strongly convex case, while effecting major savings in the communication overhead both in transmitted bits as well as in communication rounds. Empirically, experiments with real data corroborate a significant communication reduction compared to existing gradient- and stochastic gradient-based algorithms.