Enhancing HVAC Control Systems Using a Steady Soft Actor–Critic Deep Reinforcement Learning Approach

Buildings account for a substantial portion of global energy use, with about one-third of total consumption attributed to them, according to IEA statistics, significantly contributing to carbon emissions. Building energy efficiency is crucial for combating climate change and achieving energy savings. Smart buildings, leveraging intelligent control systems, optimize energy use to reduce consumption and emissions. Deep reinforcement learning (DRL) algorithms have recently gained attention for heating, ventilation, and air conditioning (HVAC) control in buildings. This paper reviews current research on DRL-based HVAC management and identifies key issues in existing algorithms. We propose an enhanced intelligent building energy management algorithm based on the Soft Actor–Critic (SAC) framework to address these challenges. Our approach employs the distributed soft policy iteration from the Distributional Soft Actor–Critic (DSAC) algorithm to improve action–state return stability. Specifically, we introduce cumulative returns into the SAC framework and recalculate target values, which reduces the loss function. The proposed HVAC control algorithm achieved 24.2% energy savings compared to the baseline SAC algorithm. This study contributes to the development of more energy-efficient HVAC systems in smart buildings, aiding in the fight against climate change and promoting energy savings.

Cette oeuvre a été publiée sous la license Creative Commons Attribution 4.0 (CC-BY 4.0). Il est autorisé de partager et adapter l'oeuvre tant que l'auteur est crédité et la license est indiquée (avec le lien ci-dessus). Vous devez aussi indiquer si des changements on été fait vis-à-vis de l'original.