Deep Learning and Reinforcement Learning for Modeling Occupants’ Information in an Occupant-Centric Building Control: A Systematic Literature Review
Publication: Construction Research Congress 2024
ABSTRACT
The Occupant-Centric Control (OCC) strategy incorporates occupant information in the building facilities control to improve energy efficiency while maintaining an acceptable level of occupant comfort. Predictive control strategies are necessary to implement OCC in complex systems like HVAC, which pose a significant challenge given the stochasticity of occupant behavior in built environments. Nonetheless, the recent advancements in Machine Learning (ML) and the Internet of Things (IoT) have made data-driven strategies more feasible in OCC of building systems. In this context, Deep Learning (DL) and Reinforcement Learning (RL) techniques have gained significant attention due to their ability to handle large volumes of data and achieve high prediction accuracy. However, the current literature lacks systematic knowledge of algorithm selection in the different OCC contexts. To address this gap, this paper presents a systematic literature review of DL and RL algorithms applied to OCC and provides organized information on the choice of algorithms by classifying occupant information into four levels based on increasing personalization. Subsequently, it identifies the algorithms suitable for each level to establish a systematic foundation for selecting DL and RL algorithms based on the degree of personalization required. The paper also highlights areas for future research in this area.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Azuatalam, D., W.-L. Lee, F. de Nijs, and A. Liebman. 2020. “Reinforcement learning for whole-building HVAC control and demand response.” Energy and AI, 2. https://doi.org/10.1016/j.egyai.2020.100020.
Bayer, D., and M. Pruckner. 2022. “Enhancing the Performance of Multi-Agent Reinforcement Learning for Controlling HVAC Systems.” 2022 IEEE Conference on Technologies for Sustainability, SusTech 2022, 187–194.
Choi, H., J. Lee, Y. Yi, H. Na, K. Kang, and T. Kim. 2022. “Deep vision-based occupancy counting: Experimental performance evaluation and implementation of ventilation control.” Build Environ, 223. https://doi.org/10.1016/j.buildenv.2022.109496.
Choi, H., H. Na, T. Kim, and T. Kim. 2021a. “Vision-based estimation of clothing insulation for building control: A case study of residential buildings.” Build Environ, 202. https://doi.org/10.1016/j.buildenv.2021.108036.
Choi, H., C. Y. Um, K. Kang, H. Kim, and T. Kim. 2021b. “Application of vision-based occupancy counting method using deep learning and performance analysis.” Energy Build, 252. https://doi.org/10.1016/j.enbuild.2021.111389.
Deng, Z., and Q. Chen. 2020. “Development and validation of a smart HVAC control system for multi-occupant offices by using occupants’ physiological signals from wristband.” Energy Build, 214. https://doi.org/10.1016/j.enbuild.2020.109872.
Deng, Z., and Q. Chen. 2021. “Reinforcement learning of occupant behavior model for cross-building transfer learning to various HVAC control systems.” Energy Build, 238. https://doi.org/10.1016/j.enbuild.2021.110860.
Ding, Z.-K., Q.-M. Fu, J.-P. Chen, H.-J. Wu, Y. Lu, and F.-Y. Hu. 2022. “Energy-efficient control of thermal comfort in multi-zone residential HVAC via reinforcement learning.” Conn Sci, 34 (1): 2364–2394. https://doi.org/10.1080/09540091.2022.2120598.
Esrafilian-Najafabadi, M., and F. Haghighat. 2021. “Occupancy-based HVAC control using deep learning algorithms for estimating online preconditioning time in residential buildings.” Energy Build, 252. https://doi.org/10.1016/j.enbuild.2021.111377.
Gupta, A., Y. Badr, A. Negahban, and R. G. Qiu. 2021. “Energy-efficient heating control for smart buildings with deep reinforcement learning.” Journal of Building Engineering, 34. https://doi.org/10.1016/j.jobe.2020.101739.
Han, M., R. May, X. Zhang, X. Wang, S. Pan, Y. Da, and Y. Jin. 2020. “A novel reinforcement learning method for improving occupant comfort via window opening and closing.” Sustain Cities Soc, 61: 102247. Elsevier. https://doi.org/10.1016/J.SCS.2020.102247.
Heidari, A., F. Maréchal, and D. Khovalyg. 2022a. “An occupant-centric control framework for balancing comfort, energy use and hygiene in hot water systems: A model-free reinforcement learning approach.” Appl Energy, 312: 118833. Elsevier. https://doi.org/10.1016/J.APENERGY.2022.118833.
Heidari, A., F. Maréchal, and D. Khovalyg. 2022b. “Reinforcement Learning for proactive operation of residential energy systems by learning stochastic occupant behavior and fluctuating solar energy: Balancing comfort, hygiene and energy use.” Appl Energy, 318: 119206. Elsevier. https://doi.org/10.1016/J.APENERGY.2022.119206.
Huang, Q., and K. Hao. 2020. “Development of cnn-based visual recognition air conditioner for smart buildings.” Journal of Information Technology in Construction, 25: 361–373. https://doi.org/10.36680/j.itcon.2020.021.
Jiang, Z., Z. Deng, X. Wang, and B. Dong. 2023. “PANDEMIC: Occupancy driven predictive ventilation control to minimize energy consumption and infection risk.” Appl Energy, 334. https://doi.org/10.1016/j.apenergy.2023.120676.
Jin, Y., D. Yan, X. Zhang, J. An, and M. Han. 2021. “A data-driven model predictive control for lighting system based on historical occupancy in an office building: Methodology development.” Build Simul, 14 (1): 219–235. https://doi.org/10.1007/s12273-020-0638-x.
Jung, S., J. Jeoung, and T. Hong. 2022. “Occupant-centered real-time control of indoor temperature using deep learning algorithms.” Build Environ, 208: 108633. Pergamon. https://doi.org/10.1016/J.BUILDENV.2021.108633.
Lei, Y., S. Zhan, E. Ono, Y. Peng, Z. Zhang, T. Hasama, and A. Chong. 2022. “A practical deep reinforcement learning framework for multivariate occupant-centric control in buildings.” Appl Energy, 324. https://doi.org/10.1016/j.apenergy.2022.119742.
Li, X., and Q. Chen. 2021. “Development of a novel method to detect clothing level and facial skin temperature for controlling HVAC systems.” Energy Build, 239. https://doi.org/10.1016/j.enbuild.2021.110859.
Liu, J., I. W. Foged, and T. B. Moeslund. 2022. “Automatic estimation of clothing insulation rate and metabolic rate for dynamic thermal comfort assessment.” Pattern Analysis and Applications, 25 (3): 619–634. https://doi.org/10.1007/s10044-021-00961-5.
Mahmud, A., E. A. Dhrubo, S. S. Ahmed, A. H. Chowdhury, M. F. Hossain, H. Rahman, and N.-A. Masood. 2022. “Energy conservation for existing cooling and lighting loads.” Energy, 255. https://doi.org/10.1016/j.energy.2022.124588.
Mata, O., J. I. Méndez, P. Ponce, T. Peffer, A. Meier, and A. Molina. 2023. “Energy Savings in Buildings Based on Image Depth Sensors for Human Activity Recognition.” Energies (Basel), 16 (3). https://doi.org/10.3390/en16031078.
Mutis, I., A. Ambekar, and V. Joshi. 2020. “Real-time space occupancy sensing and human motion analysis using deep learning for indoor air quality control.” Autom Constr, 116. https://doi.org/10.1016/j.autcon.2020.103237.
Navarro, R. C., A. R. Ruiz, F. J. V. Molina, M. J. S. Romero, J. D. Chaparro, D. V. Alises, and J. C. L. Lopez. 2022. “Indoor occupancy estimation for smart utilities: A novel approach based on depth sensors.” Build Environ, 222. https://doi.org/10.1016/j.buildenv.2022.109406.
Park, J. Y., T. Dougherty, H. Fritz, and Z. Nagy. 2019. “LightLearn: An adaptive and occupant centered controller for lighting based on reinforcement learning.” Build Environ, 147: 397–414. https://doi.org/10.1016/j.buildenv.2018.10.028.
Park, J. Y., and Z. Nagy. 2020. “HVACLearn: A reinforcement learning based occupant-centric control for thermostat set-points.” Proceedings of the Eleventh ACM International Conference on Future Energy Systems, 434–437.
Peng, Y., Z. Nagy, and A. Schlüter. 2019. “Temperature-preference learning with neural networks for occupant-centric building indoor climate controls.” Build Environ, 154: 296–308. https://doi.org/10.1016/j.buildenv.2019.01.036.
Qin, S., L. Yu, D. Yue, and C. Shen. 2021. “Optimal HVAC Control in Shared Office Spaces Based on Deep Reinforcement Learning.” Proceeding - 2021 China Automation Congress, CAC 2021, 1599–1604.
Somu, N., A. Sriram, A. Kowli, and K. Ramamritham. 2021. “A hybrid deep transfer learning strategy for thermal comfort prediction in buildings.” Build Environ, 204. https://doi.org/10.1016/j.buildenv.2021.108133.
Tien, P. W., S. Wei, T. W. Chow, J. Darkwa, C. Wood, and J. K. Calautit. 2022. “Enhancing the detection performance of a vision-based occupancy detector for buildings.” Proceedings of the Institution of Civil Engineers: Engineering Sustainability. https://doi.org/10.1680/jensu.22.00013.
Tien, P. W., S. Wei, T. Liu, J. Calautit, J. Darkwa, and C. Wood. 2021. “A deep learning approach towards the detection and recognition of opening of windows for effective management of building ventilation heat losses and reducing space heating demand.” Renew Energy, 177: 603–625. https://doi.org/10.1016/j.renene.2021.05.155.
Wang, Y., Y. Han, Y. Wu, E. Korkina, Z. Zhou, and V. Gagarin. 2022a. “An occupant-centric adaptive façade based on real-time and contactless glare and thermal discomfort estimation using deep learning algorithm.” Build Environ, 214. https://doi.org/10.1016/j.buildenv.2022.108907.
Wang, Z., J. Calautit, P. W. Tien, S. Wei, W. Zhang, Y. Wu, and L. Xia. 2023. “An occupant-centric control strategy for indoor thermal comfort, air quality and energy management.” Energy Build, 285. https://doi.org/10.1016/j.enbuild.2023.112899.
Wang, Z., J. Calautit, S. Wei, P. W. Tien, and L. Xia. 2022b. “Real-time building heat gains prediction and optimization of HVAC setpoint: An integrated framework.” Journal of Building Engineering, 49. https://doi.org/10.1016/j.jobe.2022.104103.
Wang, Z., T. Hong, and M. A. Piette. 2019. “Data fusion in predicting internal heat gains for office buildings through a deep learning approach.” Appl Energy, 240: 386–398. https://doi.org/10.1016/j.apenergy.2019.02.066.
Yayla, A., K. S. Świerczewska, M. Kaya, B. Karaca, Y. Arayıcı, Y. E. Ayözen, and O. B. Tokdemir. 2022. “Artificial Intelligence (AI)-Based Occupant-Centric Heating Ventilation and Air Conditioning (HVAC) Control System for Multi-Zone Commercial Buildings.” Sustainability (Switzerland), 14 (23). https://doi.org/10.3390/su142316107.
Yu, L., Y. Sun, Z. Xu, C. Shen, D. Yue, T. Jiang, and X. Guan. 2021. “Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings.” IEEE Trans Smart Grid, 12 (1): 407–419. https://doi.org/10.1109/TSG.2020.3011739.
Yu, L., Z. Xu, T. Zhang, X. Guan, and D. Yue. 2022. “Energy-efficient personalized thermal comfort control in office buildings based on multi-agent deep reinforcement learning.” Build Environ, 223. https://doi.org/10.1016/j.buildenv.2022.109458.
Zhang, C., Z. Zhang, and V. Loftness. 2019. “Bio-sensing and reinforcement learning approaches for occupant-centric control.” ASHRAE Trans, 364–371.
Zhuang, C., R. Choudhary, and A. Mavrogianni. 2022. “Probabilistic occupancy forecasting for risk-aware optimal ventilation through autoencoder Bayesian deep neural networks.” Build Environ, 219. https://doi.org/10.1016/j.buildenv.2022.109207.
Information & Authors
Information
Published In
Construction Research Congress 2024
Pages: 186 - 195
History
Published online: Mar 18, 2024
ASCE Technical Topics:
- Algorithms
- Architectural engineering
- Artificial intelligence and machine learning
- Bibliographies
- Building systems
- Buildings
- Computer networks
- Computer programming
- Computing in civil engineering
- Engineering fundamentals
- Information management
- Internet
- Mathematics
- Neural networks
- Structural engineering
- Structures (by type)
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.