Enhancing HVAC Control Systems Using a Steady Soft Actor–Critic Deep Reinforcement Learning Approach

Buildings account for a substantial portion of global energy use, with about one-third of total consumption attributed to them, according to IEA statistics, significantly contributing to carbon emissions. Building energy efficiency is crucial for combating climate change and achieving energy savings. Smart buildings, leveraging intelligent control systems, optimize energy use to reduce consumption and emissions. Deep reinforcement learning (DRL) algorithms have recently gained attention for heating, ventilation, and air conditioning (HVAC) control in buildings. This paper reviews current research on DRL-based HVAC management and identifies key issues in existing algorithms. We propose an enhanced intelligent building energy management algorithm based on the Soft Actor–Critic (SAC) framework to address these challenges. Our approach employs the distributed soft policy iteration from the Distributional Soft Actor–Critic (DSAC) algorithm to improve action–state return stability. Specifically, we introduce cumulative returns into the SAC framework and recalculate target values, which reduces the loss function. The proposed HVAC control algorithm achieved 24.2% energy savings compared to the baseline SAC algorithm. This study contributes to the development of more energy-efficient HVAC systems in smart buildings, aiding in the fight against climate change and promoting energy savings.

Dieses Werk wurde unter der Creative-Commons-Lizenz Namensnennung 4.0 International (CC-BY 4.0) veröffentlicht und darf unter den Lizenzbedinungen vervielfältigt, verbreitet, öffentlich zugänglich gemacht, sowie abgewandelt und bearbeitet werden. Dabei muss der Urheber bzw. Rechteinhaber genannt und die Lizenzbedingungen eingehalten werden.