Urban Climate Adaptation

Abstract

Reinforcement learning (RL)-based heating, ventilation, and air conditioning (HVAC) control serves as an innovative strategy for urban climate adaptation. However, the efficacy of such a strategy is influenced by the background climate, and its implementation may potentially alter both the indoor climate and local urban climate . This study proposes an integrated framework combining RL with an urban climate model that incorporates a building energy model, aiming to evaluate the efficacy of RL-based HVAC control across different background climates, impacts of RL strategies on indoor climate and local urban climate, and the transferability of RL strategies across cities. Our findings reveal that the reward (defined as a weighted combination of energy consumption and thermal comfort) and the impacts of RL strategies on indoor climate and local urban climate exhibit marked variability across cities with different background climates. The sensitivity of reward weights and the transferability of RL strategies are also strongly influenced by the background climate. Cities in hot climates tend to achieve higher rewards across most reward weight configurations that balance energy consumption and thermal comfort, and those cities with more varying atmospheric temperatures demonstrate greater RL strategy transferability. These findings underscore the importance of thoroughly evaluating RL-based HVAC control strategies in diverse climatic contexts . This study also provides a new insight that city-to-city learning will potentially aid the deployment of RL-based HVAC control.

Key Point: Cities with larger seasonal temperature variations tend to exhibit greater RL policy transferability

Figure 1: Transferability score of models. The color indicates the mean atmospheric forcing temperature. Size represents the variability (one standard error) of the atmospheric temperature.

In this study, an urban climate model with a BEM was employed to evaluate how RL-based HVAC control affects indoor climate and local urban climate in five cities. By using RL for HVAC control, thermal comfort can be enhanced while maintaining energy efficiency. These improvements gained from RL are closely linked to the background climate and exhibit seasonal variations for some cities. This is critical for cities aiming to implement RL-based HVAC control, as it emphasizes the needs to account for background climate when deploying such technology.

The responses of indoor climate and local urban climate to RL strategies varied notably across cities and correlated with background climates too. These findings are consistent with other studies demonstrating that the urban heat island effect and the effectiveness of urban adaptation strategies are strongly influenced by the background climate . Consequently, policymakers and stakeholders should consider developing city-specific RL strategies for HVAC control. Meanwhile, cities with larger seasonal temperature variations tend to exhibit greater transferability, meaning that RL-based control strategies trained in such cities may be more adaptable to other locations. This suggests that while RL strategies should incorporate city-specific considerations, training in a climate-diverse city could enhance their scalability across different regions.