Reinforcement Learning in Construction Engineering and Management: A Review
Publication: Journal of Construction Engineering and Management
Volume 148, Issue 11
Abstract
The construction engineering and management (CEM) domain frequently meets complex tasks due to the unavoidable complicated operation environments and the involvement of numerous workers. Being able to simulate these tasks with promised designed goals, reinforcement learning (RL) can help CEM engineers reach enhanced strategies in multi-/single-objective sequential decision-making under various sources of uncertainties. To provide a better understanding of the status quo of the RL application in CEM and its potential benefits with their strengths and limitations, this study systematically reviewed 85 CEM-related RL-based studies as a result of queries from three main scientific databases, namely Scopus, Science Direct, and Web of Science. The results of this review reveal that researchers have been increasingly applying RL methods in CEM domains, such as building energy management, infrastructure management, construction machinery, and even safety in the last few years. Our analysis showed that the reviewed papers are associated with different limitations such as generalizability, justification of selecting the approaches, and validation. This review paper alongside the presented overview of the RL methodology can assist researchers and practitioners in CEM with (1) gaining a high-level and intuitive understanding of RL algorithms, (2) identifying previous and possible future opportunities for applying RL in complex decision-making, and (3) fine tuning, proper validation, and optimizing to-be-developed RL frameworks in their future studies and applications.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
References
Aamodt, A., and E. Plaza. 1994. “Case-based reasoning: Foundational issues, methodological variations, and system approaches.” AI Commun. 7 (1): 39–59. https://doi.org/10.3233/AIC-1994-7104.
Abdelaty, A., K. J. Shrestha, and H. D. Jeong. 2020. “Estimating preconstruction services for bridge design projects.” J. Manage. Eng. 36 (4): 04020034. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000793.
Adam, B., and I. F. C. Smith. 2008a. “Active tensegrity: A control framework for an adaptive civil-engineering structure.” Comput. Struct. 86 (23–24): 2215–2223. https://doi.org/10.1016/j.compstruc.2008.05.006.
Adam, B., and I. F. C. Smith. 2008b. “Reinforcement learning for structural control.” J. Comput. Civ. Eng. 22 (2): 133–139. https://doi.org/10.1061/(ASCE)0887-3801(2008)22:2(133).
Ahn, K. U., and C. S. Park. 2020. “Application of deep Q-networks for model-free optimal control balancing between different HVAC systems.” Sci. Technol. Built Environ. 26 (1): 61–74. https://doi.org/10.1080/23744731.2019.1680234.
Akanmu, A. A., J. Olayiwola, O. Ogunseiju, and D. McFeeters. 2020. “Cyber-physical postural training system for construction workers.” Autom. Constr. 117 (Sep): 103272. https://doi.org/10.1016/j.autcon.2020.103272.
An, Y., T. Xia, R. You, D. Lai, J. Liu, and C. Chen. 2021. “A reinforcement learning approach for control of window behavior to reduce indoor concentrations in naturally ventilated buildings.” Build. Environ. 200 (Aug): 107978. https://doi.org/10.1016/j.buildenv.2021.107978.
Andriotis, C. P., and K. G. Papakonstantinou. 2019. “Managing engineering systems with large state and action spaces through deep reinforcement learning.” Reliab. Eng. Syst. Saf. 191 (Nov): 106483. https://doi.org/10.1016/j.ress.2019.04.036.
Andriotis, C. P., and K. G. Papakonstantinou. 2021. “Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints.” Reliab. Eng. Syst. Saf. 212 (Aug): 107551. https://doi.org/10.1016/j.ress.2021.107551.
Apolinarska, A. A., M. Pacher, H. Li, N. Cote, R. Pastrana, F. Gramazio, and M. Kohler. 2021. “Robotic assembly of timber joints using reinforcement learning.” Autom. Constr. 125 (May): 103569. https://doi.org/10.1016/j.autcon.2021.103569.
Arimoto, S., S. Kawamura, and F. Miyazaki. 1984. “Bettering operation of robots by learning.” J. Rob. Syst. 1 (2): 123–140. https://doi.org/10.1002/rob.4620010203.
Arora, S., and P. Doshi. 2021. “A survey of inverse reinforcement learning: Challenges, methods and progress.” Artif. Intell. 297 (Aug): 103500. https://doi.org/10.1016/j.artint.2021.103500.
Asghari, V., and S.-C. Hsu. 2021. “An open-source and extensible platform for general infrastructure asset management system.” Autom. Constr. 127 (Jul): 103692. https://doi.org/10.1016/j.autcon.2021.103692.
Asghari, V., and S.-C. Hsu. 2022. “Upscaling complex project-level infrastructure intervention planning to network assets.” J. Constr. Eng. Manage. 148 (1): 04021188. https://doi.org/10.1061/(ASCE)CO.1943-7862.0002221.
Azuatalam, D., W.-L. Lee, F. de Nijs, and A. Liebman. 2020. “Reinforcement learning for whole-building HVAC control and demand response.” Energy AI 2 (Nov): 100020. https://doi.org/10.1016/j.egyai.2020.100020.
Azulay, O., and A. Shapiro. 2021. “Wheel loader scooping controller using deep reinforcement learning.” IEEE Access 9 (Feb): 24145–24154. https://doi.org/10.1109/ACCESS.2021.3056625.
Bai, Q., S. Labi, K. C. Sinha, and P. D. Thompson. 2013. “Multiobjective optimization for project selection in network-level bridge management incorporating decision-maker’s preference using the concept of holism.” J. Bridge Eng. 18 (9): 879–889. https://doi.org/10.1061/(ASCE)BE.1943-5592.0000428.
Bellman, R. 1952. “On the theory of dynamic programming.” Proc. Natl. Acad. Sci. U.S.A. 38 (8): 716–719. https://doi.org/10.1073/pnas.38.8.716.
Bowes, B. D., A. Tavakoli, C. Wang, A. Heydarian, M. Behl, P. A. Beling, and J. L. Goodall. 2021. “Flood mitigation in coastal urban catchments using real-time stormwater infrastructure control and reinforcement learning.” J. Hydroinf. 23 (3): 529–547. https://doi.org/10.2166/hydro.2020.080.
Brandi, S., M. S. Piscitelli, M. Martellacci, and A. Capozzoli. 2020. “Deep reinforcement learning to optimise indoor temperature control and heating energy consumption in buildings.” Energy Build. 224 (Oct): 110225. https://doi.org/10.1016/j.enbuild.2020.110225.
Chembe, C., D. Kunda, I. Ahmedy, R. M. Noor, A. Q. M. Sabri, and M. A. Ngadi. 2019. “Infrastructure based spectrum sensing scheme in VANET using reinforcement learning.” Veh. Commun. 18 (Aug): 100161. https://doi.org/10.1016/j.vehcom.2019.100161.
Chemingui, Y., A. Gastli, and O. Ellabban. 2020. “Reinforcement learning-based school energy management system.” Energies (Basel) 13 (23): 6354. https://doi.org/10.3390/en13236354.
Chen, B., Z. Cai, and M. Bergés. 2019. “Gnu-RL: A precocial reinforcement learning solution for building HVAC control using a differentiable MPC policy.” In Proc., 6th ACM Int. Conf. on Systems for Energy-Efficient Buildings, Cities, and Transportation. New York: Association for Computing Machinery.
Chen, B., M. Jin, Z. Wang, T. Hong, and M. Bergés. 2020a. Towards off-policy evaluation as a prerequisite for real-world reinforcement learning in building control. New York: Association for Computing Machinery.
Chen, J., P. Tang, T. Rakstad, M. Patrick, and X. Zhou. 2020b. “Augmenting a deep-learning algorithm with canal inspection knowledge for reliable water leak detection from multispectral satellite images.” Adv. Eng. Inf. 46 (Oct): 101161. https://doi.org/10.1016/j.aei.2020.101161.
Chen, S., Y. Leng, and S. Labi. 2020c. “A deep learning algorithm for simulating autonomous driving considering prior knowledge and temporal information.” Comput.-Aided Civ. Infrastruct. Eng. 35 (4): 305–321. https://doi.org/10.1111/mice.12495.
Chen, Y., L. K. Norford, H. W. Samuelson, and A. Malkawi. 2018. “Optimal control of HVAC and window systems for natural ventilation through reinforcement learning.” Energy Build. 169 (Jun): 195–205. https://doi.org/10.1016/j.enbuild.2018.03.051.
Choi, S., D. Y. Kim, S. H. Han, and Y. H. Kwak. 2014. “Conceptual cost-prediction model for public road planning via rough set theory and case-based reasoning.” J. Constr. Eng. Manage. 140 (1): 04013026. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000743.
Clark, W. 2014. The Gantt chart: A working tool of management. Miami, FL: HardPress Publishing.
Clements, L. M., and K. M. Kockelman. 2017. “Economic effects of automated vehicles.” Transp. Res. Rec. 2606 (1): 106–114. https://doi.org/10.3141/2606-14.
Coraci, D., S. Brandi, M. S. Piscitelli, and A. Capozzoli. 2021. “Online implementation of a soft actor-critic agent to enhance indoor temperature control and energy efficiency in buildings.” Energies (Basel) 14 (4): 997. https://doi.org/10.3390/en14040997.
Costanzo, G. T., S. Iacovella, F. Ruelens, T. Leurs, and B. J. Claessens. 2016. “Experimental analysis of data-driven control for a building heating system.” Sustainable Energy Grids Networks 6 (Jun): 81–90. https://doi.org/10.1016/j.segan.2016.02.002.
Cunha, B., A. Madureira, B. Fonseca, and J. Matos. 2021. “Intelligent scheduling with reinforcement learning.” Appl. Sci. 11 (8): 3710. https://doi.org/10.3390/app11083710.
Dalamagkidis, K., D. Kolokotsa, K. Kalaitzakis, and G. S. Stavrakakis. 2007. “Reinforcement learning for energy conservation and comfort in buildings.” Build. Environ. 42 (7): 2686–2698. https://doi.org/10.1016/j.buildenv.2006.07.010.
Deng, Z., and Q. Chen. 2021. “Reinforcement learning of occupant behavior model for cross-building transfer learning to various HVAC control systems.” Energy Build. 238 (May): 110860. https://doi.org/10.1016/j.enbuild.2021.110860.
Dogan, S. Z., D. Arditi, and H. M. Günaydın. 2006. “Determining attribute weights in a CBR model for early cost prediction of structural systems.” J. Constr. Eng. Manage. 132 (10): 1092–1098. https://doi.org/10.1061/(ASCE)0733-9364(2006)132:10(1092).
Donthu, N., S. Kumar, D. Mukherjee, N. Pandey, and W. M. Lim. 2021. “How to conduct a bibliometric analysis: An overview and guidelines.” J. Bus. Res. 133 (Sep): 285–296. https://doi.org/10.1016/j.jbusres.2021.04.070.
Durango-Cohen, P. L. 2004. “Maintenance and repair decision making for infrastructure facilities without a deterioration model.” J. Infrastruct. Syst. 10 (1): 1–8. https://doi.org/10.1061/(ASCE)1076-0342(2004)10:1(1).
Eggertson, L. 2010. “Lancet retracts 12-year-old article linking autism to MMR vaccines.” Can. Med. Assoc. J. 182 (4): E199–E200. https://doi.org/10.1503/cmaj.109-3179.
Eller, L., L. C. Siafara, and T. Sauter. 2018. “Adaptive control for building energy management using reinforcement learning.” In Proc., 2018 IEEE Int. Conf. on Industrial Technology (ICIT), 1562–1567. New York: IEEE.
Elmasry, M., T. Zayed, and A. Hawari. 2019. “Multi-objective optimization model for inspection scheduling of sewer pipelines.” J. Constr. Eng. Manage. 145 (2): 04018129. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001599.
Erharter, G. H., T. F. Hansen, Z. Liu, and T. Marcher. 2021. “Reinforcement learning based process optimization and strategy development in conventional tunneling.” Autom. Constr. 127 (Jul): 103701. https://doi.org/10.1016/j.autcon.2021.103701.
Faddel, S., G. Tian, Q. Zhou, and H. Aburub. 2020. “On the performance of data-driven reinforcement learning for commercial HVAC control.” In Proc., 2020 IEEE Industry Applications Society Annual Meeting, 1–7. New York: IEEE.
Farahani, A., H. Wallbaum, and J.-O. Dalenbäck. 2020. “Cost-optimal maintenance and renovation planning in multifamily buildings with annual budget constraints.” J. Constr. Eng. Manage. 146 (3): 04020009. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001778.
Feng, K., and D. Dong. 2021. “Resource-constrained project scheduling problem under multiple time constraints.” J. Constr. Eng. Manage. 147 (2): 04020170. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001990.
Ghannad, P., Y.-C. Lee, and J. O. Choi. 2021. “Prioritizing postdisaster recovery of transportation infrastructure systems using multiagent reinforcement learning.” J. Manage. Eng. 37 (1): 04020100. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000868.
Glass, G. V., and M. L. Smith. 1979. “Meta-analysis of research on class size and achievement.” Educ. Eval. Policy Anal. 1 (1): 2–16. https://doi.org/10.3102/01623737001001002.
Goh, Y. M., and D. K. H. Chua. 2010. “Case-based reasoning approach to construction safety hazard identification: Adaptation and utilization.” J. Constr. Eng. Manage. 136 (2): 170–178. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000116.
Graham, P., N. Nikolova, and S. Sankaran. 2020. “Tension between leadership archetypes: Systematic review to inform construction research and practice.” J. Manage. Eng. 36 (1): 03119002. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000722.
Guerrero, J. I., G. Miró-Amarante, and A. Martín. 2022. “Decision support system in health care building design based on case-based reasoning and reinforcement learning.” Expert Syst. Appl. 187 (Jan): 116037. https://doi.org/10.1016/j.eswa.2021.116037.
Gupta, A., Y. Badr, A. Negahban, and R. G. Qiu. 2021. “Energy-efficient heating control for smart buildings with deep reinforcement learning.” J. Build. Eng. 34 (Feb): 101739. https://doi.org/10.1016/j.jobe.2020.101739.
Haji Hosseinloo, A., A. Ryzhov, A. Bischi, H. Ouerdane, K. Turitsyn, and M. A. Dahleh. 2020. “Data-driven control of micro-climate in buildings: An event-triggered reinforcement learning approach.” Appl. Energy 277 (Nov): 115451. https://doi.org/10.1016/j.apenergy.2020.115451.
Hasan, A., A. Ghosh, M. N. Mahmood, and M. J. Thaheem. 2021. “Scientometric review of the twenty-first century research on women in construction.” J. Manage. Eng. 37 (3): 04021004. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000887.
Hasselt, H. 2010. “Double Q-learning.” In Advances in neural information processing systems. Red Hook, NY: Curran Associates.
Henze, G., and J. Schoenmann. 2003. “Evaluation of reinforcement learning control for thermal energy storage systems.” HVAC&R Res. 9 (3): 259–275. https://doi.org/10.1080/10789669.2003.10391069.
Hodge, V. J., R. Hawkins, and R. Alexander. 2021. “Deep reinforcement learning for drone navigation using sensor data.” Neural Comput. Appl. 33 (6): 2015–2033. https://doi.org/10.1007/s00521-020-05097-x.
Ji, Y., and F. Leite. 2020. “Optimized planning approach for multiple tower cranes and material supply points using mixed-integer programming.” J. Constr. Eng. Manage. 146 (3): 04020007. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001781.
Jia, R., M. Jin, K. Sun, T. Hong, and C. Spanos. 2019. “Advanced building control via deep reinforcement learning.” Energy Procedia 158 (Feb): 6158–6163. https://doi.org/10.1016/j.egypro.2019.01.494.
Jin, H., M. Nahangi, P. M. Goodrum, and Y. Yuan. 2020. “Multiobjective optimization for scaffolding space planning in industrial piping construction using model-based simulation programming.” J. Comput. Civ. Eng. 34 (1): 06019001. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000861.
Jin, R., S. Han, C. Hyun, and J. Kim. 2014. “Improving accuracy of early stage cost estimation by revising categorical variables in a case-based reasoning model.” J. Constr. Eng. Manage. 140 (7): 04014025. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000863.
Jung, S., J. Jeoung, H. Kang, and T. Hong. 2021. “Optimal planning of a rooftop PV system using GIS-based reinforcement learning.” Appl. Energy 298 (Sep): 117239. https://doi.org/10.1016/j.apenergy.2021.117239.
Kazmi, H., F. Mehmood, S. Lodeweyckx, and J. Driesen. 2018. “Gigawatt-hour scale savings on a budget of zero: Deep reinforcement learning based optimal control of hot water systems.” Energy 144 (Feb): 159–168. https://doi.org/10.1016/j.energy.2017.12.019.
Khan, M. A.-M., M. R. J. Khan, A. Tooshil, N. Sikder, M. A. P. Mahmud, A. Z. Kouzani, and A.-A. Nahid. 2020. “A systematic review on reinforcement learning-based robotics within the last decade.” IEEE Access 8 (Sep): 176598–176623. https://doi.org/10.1109/ACCESS.2020.3027152.
Khazaeli, S., L. H. Nguyen, and J. A. Goulet. 2021. “Anomaly detection using state-space models and reinforcement learning.” Struct. Control and Health Monit. 28 (6): e2720. https://doi.org/10.1002/stc.2720.
Kim, G.-H., S.-H. An, and K.-I. Kang. 2004. “Comparison of construction cost estimating models based on regression analysis, neural networks, and case-based reasoning.” Build. Environ. 39 (10): 1235–1242. https://doi.org/10.1016/j.buildenv.2004.02.013.
Kirkpatrick, J., et al. 2017. “Overcoming catastrophic forgetting in neural networks.” Proc. Natl. Acad. Sci. 114 (13): 3521–3526. https://doi.org/10.1073/pnas.1611835114.
Knuth, D. E. 1997. The art of computer programming. Reading, MA: Addison-Wesley.
Koc, K., and A. P. Gurgun. 2021. “Stakeholder-associated life cycle risks in construction supply chain.” J. Manage. Eng. 37 (1): 04020107. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000881.
Krishna Lakshmanan, A., R. Elara Mohan, B. Ramalingam, A. Vu Le, P. Veerajagadeshwar, K. Tiwari, and M. Ilyas. 2020. “Complete coverage path planning using reinforcement learning for tetromino based cleaning and maintenance robot.” Autom. Constr. 112 (Apr): 103078. https://doi.org/10.1016/j.autcon.2020.103078.
Kurte, K., J. Munk, O. Kotevska, K. Amasyali, R. Smith, E. McKee, Y. Du, B. Cui, T. Kuruganti, and H. Zandi. 2020. “Evaluating the adaptability of reinforcement learning based HVAC control for residential houses.” Sustainability 12 (18): 7727. https://doi.org/10.3390/su12187727.
Lee, D., and M. Kim. 2021. “Autonomous construction hoist system based on deep reinforcement learning in high-rise building construction.” Autom. Constr. 128 (Aug): 103737. https://doi.org/10.1016/j.autcon.2021.103737.
Lee, J.-Y., S. Huang, A. Rahman, A. Smith, and S. Katipamula. 2020. Flexible reinforcement learning framework for building control using EnergyPlus-modelica energy models. New York: Association for Computing Machinery.
Lee, S., and D.-H. Choi. 2019. “Reinforcement learning-based energy management of smart home with rooftop solar photovoltaic system, energy storage system, and home appliances.” Sensors 19 (18): 3937. https://doi.org/10.3390/s19183937.
Levine, S., A. Kumar, G. Tucker, and J. Fu. 2020. “Offline reinforcement learning: Tutorial, review, and perspectives on open problems.” Preprint, submitted May 4, 2020. https://arxiv.org/abs/2005.01643.
Levine, S. S., and M. J. Prietula. 2014. “Open collaboration for innovation: Principles and performance.” Organ. Sci. 25 (5): 1414–1433. https://doi.org/10.1287/orsc.2013.0872.
Li, B., and L. Xia. 2015. “A multi-grid reinforcement learning method for energy conservation and comfort of HVAC in buildings.” In Proc., 2015 IEEE Int. Conf. on Automation Science and Engineering (CASE), 444–449. New York: IEEE.
Li, D., D. Zhao, Y. Zhu, and Z. Xia. 2015. “Thermal comfort control based on MEC algorithm for HVAC systems.” In Proc., 2015 Int. Joint Conf. on Neural Networks (IJCNN), 1–6. New York: IEEE.
Li, Y., K. Lu, and Y. Lu. 2017. “Project schedule forecasting for skyscrapers.” J. Manage. Eng. 33 (3): 05016023. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000498.
Liang, C.-J., V. R. Kamat, and C. C. Menassa. 2020. “Teaching robots to perform quasi-repetitive construction tasks through human demonstration.” Autom. Constr. 120 (Dec): 103370. https://doi.org/10.1016/j.autcon.2020.103370.
Lissa, P., C. Deane, M. Schukat, F. Seri, M. Keane, and E. Barrett. 2021. “Deep reinforcement learning for home energy management system control.” Energy AI 3 (Mar): 100043. https://doi.org/10.1016/j.egyai.2020.100043.
Liu, D., J. Cao, and X. Lei. 2019. “Slabstone installation skill acquisition for dual-arm robot based on reinforcement learning.” In Proc., 2019 IEEE Int. Conf. on Robotics and Biomimetics (ROBIO), 1298–1305. New York: IEEE.
Liu, J., P. Liu, L. Feng, W. Wu, D. Li, and F. Chen. 2020a. “Towards automated clash resolution of reinforcing steel design in reinforced concrete frames via Q-learning and building information modeling.” Autom. Constr. 112 (Apr): 103062. https://doi.org/10.1016/j.autcon.2019.103062.
Liu, J., and M. Lu. 2018. “Constraint programming approach to optimizing project schedules under material logistics and crew availability constraints.” J. Constr. Eng. Manage. 144 (7): 04018049. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001507.
Liu, S., and G. P. Henze. 2006. “Experimental analysis of simulated reinforcement learning control for active and passive building thermal storage inventory: Part 1. Theoretical foundation.” Energy Build. 38 (2): 142–147. https://doi.org/10.1016/j.enbuild.2005.06.002.
Liu, S., and G. P. Henze. 2007. “Evaluation of reinforcement learning for optimal control of building active and passive thermal storage inventory.” J. Sol. Energy Eng. 129 (2): 215–225. https://doi.org/10.1115/1.2710491.
Liu, T., Z. Tan, C. Xu, H. Chen, and Z. Li. 2020b. “Study on deep reinforcement learning techniques for building energy consumption forecasting.” Energy Build. 208 (Feb): 109675. https://doi.org/10.1016/j.enbuild.2019.109675.
Lu, S., W. Wang, C. Lin, and E. C. Hameen. 2019. “Data-driven simulation of a thermal comfort-based temperature set-point control with ASHRAE RP884.” Build. Environ. 156 (Jun): 137–146. https://doi.org/10.1016/j.buildenv.2019.03.010.
Lu, Z., L. Xuan, and D. Sunil. 2021. “A reinforcement learning-based stakeholder value aggregation model for collaborative decision making on disaster resilience.” In Proc., Computing in Civil Engineering 2019. Reston, VA: ASCE.
Mahdavian, A., A. Shojaei, M. Salem, J. S. Yuan, and A. A. Oloufa. 2021. “Data-driven predictive modeling of highway construction cost items.” J. Constr. Eng. Manage. 147 (3): 04020180. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001991.
Mandow, L., J.-L. Pérez-de-la-Cruz, A. B. Rodríguez-Gavilán, and M. Ruiz-Montiel. 2020. “Architectural planning with shape grammars and reinforcement learning: Habitability and energy efficiency.” Eng. Appl. Artif. Intell. 96 (Nov): 103909. https://doi.org/10.1016/j.engappai.2020.103909.
Memarzadeh, M., and M. Pozzi. 2019. “Model-free reinforcement learning with model-based safe exploration: Optimizing adaptive recovery process of infrastructure systems.” Struct. Saf. 80 (Sep): 46–55. https://doi.org/10.1016/j.strusafe.2019.04.003.
Memarzadeh, M., M. Pozzi, and Z. J. Kolter. 2015. “Optimal planning and learning in uncertain environments for the management of wind farms.” J. Comput. Civ. Eng. 29 (5): 04014076. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000390.
Mnih, V., A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, and D. Silver. 2016. “Asynchronous methods for deep reinforcement learning.” In Proc., 33rd Int. Conf. on Machine Learning (ICML 2016), edited by M. F. Balcan and K. Q. Weinberger, 1928–1937. Cambridge, MA: The Journal of Machine Learning Research.
Mnih, V., K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller. 2013. “Playing Atari with deep reinforcement learning.” Preprint, submitted December 19, 2013. https://arxiv.org/abs/1312.5602.
Mostafa, K., and T. Hegazy. 2021. “Review of image-based analysis and applications in construction.” Autom. Constr. 122 (Feb): 103516. https://doi.org/10.1016/j.autcon.2020.103516.
Mullapudi, A., M. J. Lewis, C. L. Gruden, and B. Kerkez. 2020. “Deep reinforcement learning for the real time control of stormwater systems.” Adv. Water Resour. 140 (Jun): 103600. https://doi.org/10.1016/j.advwatres.2020.103600.
Murugesan, S., Z. Jiang, M. J. Risbeck, J. Amores, C. Zhang, V. Ramamurti, K. H. Drees, and Y. M. Lee. 2020. “Less is more: Simplified state-action space for deep reinforcement learning based HVAC control.” In Proc., 1st Int. Workshop on Reinforcement Learning for Energy Management in Buildings and Cities, RLEM 2020, 20–23. Santa Clara, CA: Association for Computing Machinery.
Mushtaq, A., I. U. Haq, M. U. Imtiaz, A. Khan, and O. Shafiq. 2021. “Traffic flow management of autonomous vehicles using deep reinforcement learning and smart rerouting.” IEEE Access 9 (Mar): 51005–51019. https://doi.org/10.1109/ACCESS.2021.3063463.
Naoum, S. G., J. Harris, J. Rizzuto, and C. Egbu. 2020. “Gender in the construction industry: Literature review and comparative survey of men’s and women’s perceptions in UK construction consultancies.” J. Manage. Eng. 36 (2): 04019042. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000731.
Naug, A., I. Ahmed, and G. Biswas. 2019. “Online energy management in commercial buildings using deep reinforcement learning.” In Proc., 2019 IEEE Int. Conf. on Smart Computing (SMARTCOMP), 249–257. New York: IEEE.
Nehal, E., A. Tarek, and M. Mohamed. 2020. “Application of evolutionary optimization algorithms for rehabilitation of water distribution networks.” J. Constr. Eng. Manage. 146 (7): 04020069. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001856.
Nguyen, H., and H. La. 2019. “Review of deep reinforcement learning for robot manipulation.” In Proc., 2019 3rd IEEE Int. Conf. on Robotic Computing (IRC), 590–595. New York: IEEE.
Nian, R., J. Liu, and B. Huang. 2020. “A review on reinforcement learning: Introduction and applications in industrial process control.” Comput. Chem. Eng. 139 (Aug): 106886. https://doi.org/10.1016/j.compchemeng.2020.106886.
Nozhati, S., B. R. Ellingwood, and E. K. P. Chong. 2020. “Stochastic optimal control methodologies in risk-informed community resilience planning.” Struct. Saf. 84 (May): 101920. https://doi.org/10.1016/j.strusafe.2019.101920.
Oroojlooy Jadid, A., and D. Hajinezhad. 2019. A review of cooperative multi-agent deep reinforcement learning. Ithaca, NY: arXiv. https://doi.org/10.48550/arXiv.1908.03963.
Pan, Y., and L. Zhang. 2021. “Roles of artificial intelligence in construction engineering and management: A critical review and future trends.” Autom. Constr. 122 (Feb): 103517. https://doi.org/10.1016/j.autcon.2020.103517.
Papakonstantinou, K. G., and M. Shinozuka. 2014. “Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part II: POMDP implementation.” Reliab. Eng. Syst. Saf. 130 (Oct): 214–224. https://doi.org/10.1016/j.ress.2014.04.006.
Park, J. Y., T. Dougherty, H. Fritz, and Z. Nagy. 2019. “LightLearn: An adaptive and occupant centered controller for lighting based on reinforcement learning.” Build. Environ. 147 (Jan): 397–414. https://doi.org/10.1016/j.buildenv.2018.10.028.
Park, J. Y., and Z. Nagy. 2020. HVACLearn: A reinforcement learning based occupant-centric control for thermostat set-points. New York: Association for Computing Machinery.
Park, S., S. Park, M. Choi, S. Lee, T. Lee, S. Kim, C. Keonhee, and S. Park. 2020. “Reinforcement learning-based BEMS architecture for energy usage optimization.” Sensors 20 (17): 4918. https://doi.org/10.3390/s20174918.
Pereira, E., S. Han, and S. Abourizk. 2018. “Integrating case-based reasoning and simulation modeling for testing strategies to control safety performance.” J. Comput. Civ. Eng. 32 (6): 04018047. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000792.
Petro, Y., U. Ojiako, T. Williams, and A. Marshall. 2019. “Organizational ambidexterity: A critical review and development of a project-focused definition.” J. Manage. Eng. 35 (3): 03119001. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000685.
Renard, S., B. Corbett, and O. Swei. 2021. “Minimizing the global warming impact of pavement infrastructure through reinforcement learning.” Resour. Conserv. Recycl. 167 (Apr): 105240. https://doi.org/10.1016/j.resconrec.2020.105240.
Ruiz-Montiel, M., J. Boned, J. Gavilanes, E. Jiménez, L. Mandow, and J.-L. Pérez-de-la-Cruz. 2013. “Design with shape grammars and reinforcement learning.” Adv. Eng. Inf. 27 (2): 230–245. https://doi.org/10.1016/j.aei.2012.12.004.
Rummery, G. A., and M. Niranjan. 1994. On-line Q-learning using connectionist systems. Cambridge, UK: Univ. of Cambridge, Dept. of Engineering.
Russell, S. J., and P. Norvig. 2016. Artificial intelligence: A modern approach. Upper Saddle River, NJ: Prentice Hall.
Sahachaisaree, S., P. Sae-ma, and P. Nanakorn. 2020. “Two-dimensional truss topology design by reinforcement learning.” In Proc., ICSCEA 2019, 1237–1245. Singapore: Springer.
Sahari Moghaddam, A., E. Rezazadeh Azar, Y. Mejias, and H. Bell. 2020. “Estimating stripping of asphalt coating using -means clustering and machine learning–based classification.” J. Comput. Civ. Eng. 34 (1): 04019044. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000864.
Sai, P. G., M. Sanayei, and I. F. Smith. 2021. “Model-class selection using clustering and classification for structural identification and prediction.” J. Comput. Civ. Eng. 35 (1): 04020051. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000932.
Schaul, T., J. Quan, I. Antonoglou, and D. Silver. 2016. “Prioritized experience replay.” Preprint, submitted November 18, 2015. https://arxiv.org/abs/1511.05952.
Schreiber, T., S. Eschweiler, M. Baranski, and D. Müller. 2020. “Application of two promising Reinforcement Learning algorithms for load shifting in a cooling supply system.” Energy Build. 229 (Dec): 110490. https://doi.org/10.1016/j.enbuild.2020.110490.
Schreiber, T., C. Netsch, M. Baranski, and D. Müller. 2021. “Monitoring data-driven reinforcement learning controller training: A comparative study of different training strategies for a real-world energy system.” Energy Build. 239 (May): 110856. https://doi.org/10.1016/j.enbuild.2021.110856.
Schulman, J., F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. 2017. “Proximal policy optimization algorithms.” Preprint, submitted July 20, 2017. https://arxiv.org/abs/1707.06347.
Serrano, W. 2019. “Deep reinforcement learning algorithms in intelligent infrastructure.” Infrastructures (Basel) 4 (3): 52. https://doi.org/10.3390/infrastructures4030052.
Shi, F., R. K. Soman, J. Han, and J. K. Whyte. 2020. “Addressing adjacency constraints in rectangular floor plans using Monte-Carlo Tree Search.” Autom. Constr. 115 (Jul): 103187. https://doi.org/10.1016/j.autcon.2020.103187.
Shi, Y., J. Du, C. R. Ahn, and E. Ragan. 2019. “Impact assessment of reinforced learning methods on construction workers’ fall risk behavior using virtual reality.” Autom. Constr. 104 (Aug): 197–214. https://doi.org/10.1016/j.autcon.2019.04.015.
Shin, J., T. A. Badgwell, K.-H. Liu, and J. H. Lee. 2019. “Reinforcement learning—Overview of recent progress and implications for process control.” Comput. Chem. Eng. 127 (Aug): 282–294. https://doi.org/10.1016/j.compchemeng.2019.05.029.
Si, C., Y. Tao, J. Qiu, S. Lai, and J. Zhao. 2021. “Deep reinforcement learning based home energy management system with devices operational dependencies.” Int. J. Mach. Learn. Cybern. 12 (6): 1687–1703. https://doi.org/10.1007/s13042-020-01266-5.
Silver, D., et al. 2016. “Mastering the game of Go with deep neural networks and tree search.” Nature 529 (7587): 484–489. https://doi.org/10.1038/nature16961.
Socher, R., C. C.-Y. Lin, C. D. Manning, and A. Y. Ng. 2011. “Parsing natural scenes and natural language with recursive neural networks.” In Vol. 5781 of Proc., 28th Int. Conf. on Machine Learning (ICML-11), 129–146. https://doi.org/10.1007/978-3-642-04180-8.
Sun, Z., C. Zhang, J. Chen, P. Tang, and A. Yilmaz. 2020. “Predictive nuclear power plant outage control through computer vision and data-driven simulation.” Prog. Nucl. Energy 127 (Sep): 103448. https://doi.org/10.1016/j.pnucene.2020.103448.
Sun, Z., C. Zhang, P. Tang, Y. Wang, and Y. Liu. 2019. “Bayesian network modeling of airport runway incursion occurring processes for predictive accident control.” In Proc., Advances in Informatics and Computing in Civil and Construction Engineering: Proc. of the 35th CIB W78 2018 Conf.—IT in Design, Construction, and Management, 669–676. Chicago: International Council for Research and Innovation in Building Construction.
Sung, I., B. Choi, and P. Nielsen. 2020. “Reinforcement learning for resource constrained project scheduling problem with activity iterations and crashing.” IFAC-PapersOnLine 53 (2): 10493–10497. https://doi.org/10.1016/j.ifacol.2020.12.2794.
Sutton, R. S., and A. G. Barto. 2018. Reinforcement learning: An introduction. Cambridge, MA: A Bradford Book.
Tabandeh, A., and P. Gardoni. 2015. “Empirical Bayes approach for developing hierarchical probabilistic predictive models and its application to the seismic reliability analysis of FRP-retrofitted RC bridges.” ASCE-ASME J. Risk Uncertainty Eng. Syst. Part A: Civ. Eng. 1 (2): 04015002. https://doi.org/10.1061/AJRUA6.0000817.
Taha, H. A. 2017. Operations research: An introduction. Harlow, England: Pearson.
Tan, T., G. Mills, E. Papadonikolaki, and Z. Liu. 2021. “Combining multi-criteria decision making (MCDM) methods with building information modelling (BIM): A review.” Autom. Constr. 121 (Jan): 103451. https://doi.org/10.1016/j.autcon.2020.103451.
Valladares, W., M. Galindo, J. Gutiérrez, W.-C. Wu, K.-K. Liao, J.-C. Liao, K.-C. Lu, and C.-C. Wang. 2019. “Energy optimization associated with thermal comfort and indoor air control via a deep reinforcement learning algorithm.” Build. Environ. 155 (May): 105–117. https://doi.org/10.1016/j.buildenv.2019.03.038.
van Eck, N. J., and L. Waltman. 2010. “Software survey: VOSviewer, a computer program for bibliometric mapping.” Scientometrics 84 (2): 523–538. https://doi.org/10.1007/s11192-009-0146-3.
Van Hasselt, H., A. Guez, and D. Silver. 2015. “Deep reinforcement learning with double Q-learning.” In Vol. 30 of Proc., AAAI Conf. on Artificial Intelligence. Palo Alto, CA: Association for the Advancement of Artificial Intelligence Press.
Vázquez-Canteli, J., J. Kämpf, and Z. Nagy. 2017. “Balancing comfort and energy consumption of a heat pump using batch reinforcement learning with fitted Q-iteration.” Energy Procedia 122 (Sep): 415–420. https://doi.org/10.1016/j.egypro.2017.07.429.
Vázquez-Canteli, J. R., and Z. Nagy. 2019. “Reinforcement learning for demand response: A review of algorithms and modeling techniques.” Appl. Energy 235 (Feb): 1072–1089. https://doi.org/10.1016/j.apenergy.2018.11.002.
Wang, R. D., T. Zayed, W. Pan, S. Zheng, and S. Tariq. 2021. “A system boundary-based critical review on crane selection in building construction.” Autom. Constr. 123 (Mar): 103520. https://doi.org/10.1016/j.autcon.2020.103520.
Wang, Y., and P. Tang. 2021. “Spatiotemporal data-driven simulation and clustering of ground operations of aircraft for comprehending airport jams and collisions.” In Proc., Construction Research Congress 2020. Reston, VA: ASCE.
Wang, Y., K. Velswamy, and B. Huang. 2017. “A long-short term memory recurrent neural network based reinforcement learning controller for office heating ventilation and air conditioning systems.” Processes 5 (3): 46. https://doi.org/10.3390/pr5030046.
Wang, Z., H. Hu, and J. Gong. 2018. “Modeling worker competence to advance precast production scheduling optimization.” J. Constr. Eng. Manage. 144 (11): 04018098. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001556.
Wang, Z., T. Schaul, M. Hessel, H. Hasselt, M. Lanctot, and N. Freitas. 2015. “Dueling network architectures for deep reinforcement learning.” In Proc., Int. Conf. on Machine Learning, 1995–2003. Cambridge, MA: Journal of Machine Learning Research.
Wei, P., S. Xia, R. Chen, J. Qian, C. Li, and X. Jiang. 2020a. “A deep-reinforcement-learning-based recommender system for occupant-driven energy optimization in commercial buildings.” IEEE Internet Things J. 7 (7): 6402–6413. https://doi.org/10.1109/JIOT.2020.2974848.
Wei, S., Y. Bao, and H. Li. 2020b. “Optimal policy for structure maintenance: A deep reinforcement learning framework.” Struct. Saf. 83 (Mar): 101906. https://doi.org/10.1016/j.strusafe.2019.101906.
Wei, T., Y. Wang, and Q. Zhu. 2017. “Deep reinforcement learning for building HVAC control.” In Proc., 2017 54th ACM/EDAC/IEEE Design Automation Conf. (DAC), 1–6. New York: Association for Computing Machinery.
Wohlin, C. 2014. “Guidelines for snowballing in systematic literature studies and a replication in software engineering.” In Proc., 18th Int. Conf. on Evaluation and Assessment in Software Engineering, EASE ’14. New York: Association for Computing Machinery.
Woo, J. H., B. Kim, S. Ju, and Y. I. Cho. 2021. “Automation of load balancing for Gantt planning using reinforcement learning.” Eng. Appl. Artif. Intell. 101 (May): 104226. https://doi.org/10.1016/j.engappai.2021.104226.
Xia, K., C. Sacco, M. Kirkpatrick, C. Saidy, L. Nguyen, A. Kircaliali, and R. Harik. 2021. “A digital twin to train deep reinforcement learning agent for smart manufacturing plants: Environment, interfaces and intelligence.” J. Manuf. Syst. 58 (Jan): 210–230. https://doi.org/10.1016/j.jmsy.2020.06.012.
Xiang, Z., Y. Bao, Z. Tang, and H. Li. 2020. “Deep reinforcement learning-based sampling method for structural reliability assessment.” Reliab. Eng. Syst. Saf. 199 (Jul): 106901. https://doi.org/10.1016/j.ress.2020.106901.
Xu, Y., Y. Zhou, P. Sekula, and L. Ding. 2021. “Machine learning in construction: From shallow to deep learning.” Dev. Built Environ. 6 (May): 100045. https://doi.org/10.1016/j.dibe.2021.100045.
Yang, L., Z. Nagy, P. Goffin, and A. Schlueter. 2015. “Reinforcement learning for optimal control of low exergy buildings.” Appl. Energy 156 (Oct): 577–586. https://doi.org/10.1016/j.apenergy.2015.07.050.
Yao, L., Q. Dong, J. Jiang, and F. Ni. 2020. “Deep reinforcement learning for long-term pavement maintenance planning.” Comput.-Aided Civ. Infrastruct. Eng. 35 (11): 1230–1245. https://doi.org/10.1111/mice.12558.
Ye, H., G. Y. Li, and B.-H. F. Juang. 2019. “Deep reinforcement learning based resource allocation for V2V communications.” IEEE Trans. Veh. Technol. 68 (4): 3163–3173. https://doi.org/10.1109/TVT.2019.2897134.
Ye, Y., D. Qiu, H. Wang, T. Yi, and G. Strbac. 2021. “Real-time autonomous residential demand response management based on twin delayed deep deterministic policy gradient learning.” Energies (Basel) 14 (3): 531. https://doi.org/10.3390/en14030531.
Yehia, A. 2020. “Understanding uncertainty: A reinforcement learning approach for project-level pavement management systems.” Ph.D. thesis, Dept. of Civil Engineering, Univ. of British Columbia.
Yoon, Y. R., and H. J. Moon. 2019. “Performance based thermal comfort control (PTCC) using deep reinforcement learning for space cooling.” Energy Build. 203 (Nov): 109420. https://doi.org/10.1016/j.enbuild.2019.109420.
Yu, Z., and A. Dexter. 2010. “Online tuning of a supervisory fuzzy controller for low-energy building system using reinforcement learning.” Control Eng. Pract. 18 (5): 532–539. https://doi.org/10.1016/j.conengprac.2010.01.018.
Zhang, S., H. Yao, and S. Whiteson. 2021. Breaking the deadly triad with a target network. Cambridge, MA: Journal of Machine Learning Research.
Zhang, W., J. Wang, Y. Liu, G. Gao, S. Liang, and H. Ma. 2020. “Reinforcement learning-based intelligent energy management architecture for hybrid construction machinery.” Appl. Energy 275 (Oct): 115401. https://doi.org/10.1016/j.apenergy.2020.115401.
Zhang, Z., A. Chong, Y. Pan, C. Zhang, and K. P. Lam. 2019. “Whole building energy model for HVAC optimal control: A practical framework based on deep reinforcement learning.” Energy Build. 199 (Sep): 472–490. https://doi.org/10.1016/j.enbuild.2019.07.029.
Zhang, Z., and K. Lam. 2018. Practical implementation and evaluation of deep reinforcement learning control for a radiant heating system. New York: Association for Computing Machinery.
Zhao, H., J. Zhao, T. Shu, and Z. Pan. 2021. “Hybrid-model-based deep reinforcement learning for heating, ventilation, and air-conditioning control.” Front. Energy Res. 8 (Feb): 610518. https://doi.org/10.3389/fenrg.2020.610518.
Zhou, M., Z. Wan, H. Wang, M. Wen, R. Wu, Y. Wen, Y. Yang, W. Zhang, and J. Wang. 2021. “MALib: A parallel framework for population-based multi-agent reinforcement learning.” Preprint, submitted June 5, 2021. https://arxiv.org/abs/2106.07551.
Zhu, Q., and Q. Wang. 2017. “Real-time energy management controller design for a hybrid excavator using reinforcement learning.” J. Zhejiang Univ.-Sci. A 18 (11): 855–870. https://doi.org/10.1631/jzus.A1600650.
Zou, Z., X. Yu, and S. Ergan. 2020. “Towards optimal control of air handling units using deep reinforcement learning and recurrent neural network.” Build. Environ. 168 (Jan): 106535. https://doi.org/10.1016/j.buildenv.2019.106535.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 148 • Issue 11 • November 2022
Copyright
© 2022 American Society of Civil Engineers.
History
Published online: Sep 8, 2022
Published in print: Nov 1, 2022
Discussion open until: Feb 8, 2023
ASCE Technical Topics:
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Charles T. Jahren, Patrick X. W. Zou, John E. Taylor, Edward J. Jaselskis, Second Special Collection on Research Methodologies in Construction Engineering and Management, Journal of Construction Engineering and Management, 10.1061/JCEMD4.COENG-13129, 149, 2, (2023).
- Benjamin I. Oluleye, Daniel W.M. Chan, Prince Antwi-Afari, Adopting Artificial Intelligence for enhancing the implementation of systemic circularity in the construction industry: A critical review, Sustainable Production and Consumption, 10.1016/j.spc.2022.12.002, 35, (509-524), (2023).