Model-Free Control of Thermostatically Controlled Loads Connected to a District Heating Network
This addresses scalability and model identification challenges in district heating control, offering a practical solution for energy management, though it is incremental as it combines existing reinforcement learning and multi-agent methods.
The paper tackled the problem of optimally controlling many thermostatically controlled loads in a district heating network under uncertainty, achieving a performance within 65% of a theoretical lower bound on cost after 60 days of learning for energy arbitrage and peak shaving objectives.
Optimal control of thermostatically controlled loads connected to a district heating network is considered a sequential decision- making problem under uncertainty. The practicality of a direct model-based approach is compromised by two challenges, namely scalability due to the large dimensionality of the problem and the system identification required to identify an accurate model. To help in mitigating these problems, this paper leverages on recent developments in reinforcement learning in combination with a market-based multi-agent system to obtain a scalable solution that obtains a significant performance improvement in a practical learning time. The control approach is applied on a scenario comprising 100 thermostatically controlled loads connected to a radial district heating network supplied by a central combined heat and power plant. Both for an energy arbitrage and a peak shaving objective, the control approach requires 60 days to obtain a performance within 65% of a theoretical lower bound on the cost.