Dynamic Real-Time Optimization of Modular Unit Allocation to Off-Site Facilities in Postdisaster Reconstruction Using Deep Reinforcement Learning
Publication: Journal of Management in Engineering
Volume 40, Issue 4
Abstract
Postdisaster housing reconstruction (PDHR) requires robust, efficient planning, and coordination among dispersed prefabrication facilities and jobsites to maximize prefabrication benefits. Manual construction methods and inherent risks often lead to unforeseen incidents and delays. Previous off-site construction studies focused on specific factors, neglecting uncertainties and the improvement of the Social Vulnerability Index (SoVI). Considering all critical parameters, this study proposes a real-time optimized allocation using deep reinforcement learning (DRL) for PDHR projects. The framework employs the Q-learning algorithm to generate the best real-time schedules for each prefabrication facility based on a current work status to complete a planned project. This is illustrated through a case study with ten project sites, four types of module layouts, and four prefabrication facilities. To demonstrate the superiority of the DRL-based method, the model was compared to the Monte Carlo Simulation and the Genetic Algorithm (GA) for the nine criteria related to time, cost, and social vulnerability. The DRL-based algorithm optimized the time parameter by reducing delay by 33.6% compared to the Monte Carlo simulation and by 46.4% compared to the GA. Similarly, it reduced missed deadlines by 35.1% compared to the Monte Carlo Simulation and by 13.3% compared to the GA. When comparing the cost parameter, the DRL model reduced overall cost by 3.4% compared to the Monte Carlo simulation and by 18.6% compared to the GA. In addition, it was able to prioritize the more vulnerable jobsites over less vulnerable ones to reduce delays and missed deadlines. In this regard, the proposed approach contributes to the body of knowledge by introducing a new automated model for PDHR project work distribution, considering productivity, cost, time, resources, uncertainties, and SoVI, thereby improving informed decision-making and overall project performance.
Practical Applications
This research presents an artificial intelligence (AI)-powered system tailored for managing large construction projects, especially crucial in postdisaster rebuilding efforts. By harnessing advanced AI techniques, this system streamlines operations in prefabrication factories, ensuring timely and cost-effective completion while considering critical social needs. The key factors that set this approach apart are adaptability and continuous learning. By constantly analyzing and learning from real-time project data, the system dynamically allocates resources, minimizing delays and cost overruns while also prioritizing areas with higher social vulnerability. However, it’s important to note that the system’s effectiveness relies on specific factory layouts and the availability of up-to-date information. In practical terms, implementing this system could revolutionize how construction projects are managed. It offers real-time insights, enabling informed decision-making, streamlined operations, and enhanced collaboration among all involved stakeholders. Beyond construction, the potential applications of this technology extend to thoughtful urban planning and expediting recovery in disaster-stricken areas. Despite existing challenges, this research paves the way for AI-driven solutions that not only optimize project efficiency but also integrate social responsibility into decision-making processes. It sets a promising path toward a future where AI plays a pivotal role in optimizing projects while concurrently addressing the critical needs of society.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
The authors would like to acknowledge Boxy Construction and its owner, Mr. Mateo Atwi, for their invaluable support and cooperation throughout the course of this research endeavor. Their expertise and assistance have been instrumental in the successful execution of this project.
References
Adeli, H., and A. Karim. 2000. “Fuzzy-wavelet RBFNN model for freeway incident detection.” J. Transp. Eng. 126 (6): 464–471. https://doi.org/10.1061/(ASCE)0733-947X(2000)126:6(464).
Al-Tabtabai, H. M. 1996. “Computer-aided conflict resolution in construction management and planning.” In Proc., Int. Symp. for the Organization and Management of Construction: Managing Construction Information, 206. New York: Taylor & Francis.
Alvanchi, A., R. Azimi, S. Lee, S. M. AbouRizk, and P. Zubick. 2012. “Off-site construction planning using discrete event simulation.” J. Archit. Eng. 18 (2): 114–122. https://doi.org/10.1061/(ASCE)AE.1943-5568.0000055.
Arashpour, M., and M. Arashpour. 2015. “Analysis of workflow variability and its impacts on productivity and performance in the construction of multistory buildings.” J. Manage. Eng. 31 (6): 04015006. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000363.
Aydin, M. E., and E. Öztemel. 2000. “Dynamic job-shop scheduling using reinforcement learning agents.” Rob. Auton. Syst. 33 (2–3): 169–178. https://doi.org/10.1016/S0921-8890(00)00087-7.
Azimi, R., S. Lee, and S. M. AbouRizk. 2012. “Applying basic control theory principles to project control: Case study of off-site construction shops.” J. Comput. Civ. Eng. 26 (6): 681–690. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000190.
Bertram, N., S. Fuchs, J. Mischke, R. Palter, G. Strube, and J. Woetzel. 2019. “Modular construction: From projects to products.” In Capital projects & infrastructure, 1–34. Chicago: McKinsey & Company.
Bertsekas, D., and J. N. Tsitsiklis. 1996. Neuro-dynamic programming. Nashua, NH: Athena Scientific.
Bertsekas, D. P. 2012. “Dynamic programming and optimal control.” In Approximate dynamic programming, 4th ed. Nashua, NH: Athena Scientific.
Bris, P., and F. Bendito. 2019. “Impact of Japanese postdisaster temporary housing areas’(THAs) design on mental and social health.” Int. J. Environ. Res. Public Health 16 (23): 4757. https://doi.org/10.3390/ijerph16234757.
Bunni, N. G. 2003. Risk and insurance in construction. London: Routledge.
Chen, C. L., R. V. Neppalli, and N. Aljaber. 1996. “Genetic algorithms applied to the continuous flow shop problem.” Comput. Ind. Eng. 30 (4): 919–929. https://doi.org/10.1016/0360-8352(96)00042-3.
Cutter, S. L., and C. Finch. 2008. “Temporal and spatial changes in social vulnerability to natural hazards.” Proc. Natl. Acad. Sci. 105 (7): 2301–2306.
Deb, K., A. Pratap, S. Agarwal, and T. A. M. T. Meyarivan. 2002. “A fast and elitist multi-objective genetic algorithm: NSGA-II.” IEEE Trans. Evol. Comput. 6 (2): 182–197. https://doi.org/10.1109/4235.996017.
Deckro, R. F., E. P. Winkofsky, J. E. Hebert, and R. Gagnon. 1991. “A decomposition approach to multi-project scheduling.” Eur. J. Oper. Res. 51 (1): 110–118. https://doi.org/10.1016/0377-2217(91)90150-T.
Deria, A., P. Ghannad, and Y. C. Lee. 2020. “Evaluating implications of flood vulnerability factors with respect to income levels for building long-term disaster resilience of low-income communities.” Int. J. Disaster Risk Reduct. 48 (Sep): 101608. https://doi.org/10.1016/j.ijdrr.2020.101608.
Faghihi, V., A. Nejat, K. F. Reinschmidt, and J. H. Kang. 2015. “Automation in construction scheduling: A review of the literature.” Int. J. Adv. Manuf. Technol. 81 (Dec): 1845–1856. https://doi.org/10.1007/s00170-015-7339-0.
Feng, C.-W., L. Liu, and S. A. Burns. 1997. “Using genetic algorithms to solve construction time-cost trade-off problems.” J. Comput. Civ. Eng. 11 (3): 184–189. https://doi.org/10.1061/(ASCE)0887-3801(1997)11:3(184).
Gabel, T., and M. Riedmiller. 2012. “Distributed policy search reinforcement learning for job-shop scheduling tasks.” Int. J. Prod. Res. 50 (1): 41–61. https://doi.org/10.1080/00207543.2011.571443.
Geurtsen, M., I. Adan, and Z. Atan. 2023. “Deep reinforcement learning for optimal planning of assembly line maintenance.” J. Manuf. Syst. 69 (Aug): 170–188. https://doi.org/10.1016/j.jmsy.2023.05.011.
Ghannad, P., Y. C. Lee, and J. O. Choi. 2020a. “Feasibility and implications of the modular construction approach for rapid postdisaster recovery.” Int. J. Industrialized Constr. 1 (1): 64–75. https://doi.org/10.29173/ijic220.
Ghannad, P., Y.-C. Lee, and J. O. Choi. 2021. “Prioritizing postdisaster recovery of transportation infrastructure systems using multiagent reinforcement learning.” J. Manage. Eng. 37 (1): 04020100. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000868.
Ghannad, P., Y.-C. Lee, C. J. Friedland, J. O. Choi, and E. Yang. 2020b. “Multiobjective optimization of postdisaster reconstruction processes for ensuring long-term socioeconomic benefits.” J. Manage. Eng. 36 (4): 04020038. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000799.
Ginigaddara, B., S. Perera, Y. Feng, and P. Rahnamayiezekavat. 2022. “Development of an offsite construction typology: A Delphi study.” Buildings 12 (1): 20. https://doi.org/10.3390/buildings12010020.
Golpayegani, S. A. H., and F. Parvaresh. 2011. “The logical precedence network planning of projects, considering the finish-to-start (FS) relations, using neural networks.” Int. J. Adv. Manuf. Technol. 55 (Aug): 1123–1133. https://doi.org/10.1007/s00170-010-3125-1.
Gonçalves, J. F., J. J. Mendes, and M. G. Resende. 2008. “A genetic algorithm for the resource constrained multi-project scheduling problem.” Eur. J. Oper. Res. 189 (3): 1171–1190. https://doi.org/10.1016/j.ejor.2006.06.074.
Gunawardena, T., T. Ngo, P. Mendis, L. Aye, and R. Crawford. 2014. “Time-efficient postdisaster housing reconstruction with prefabricated modular structures.” Open House Int. 39 (3): 59–69. https://doi.org/10.1108/OHI-03-2014-B0007.
Hussein, M., A. E. Eltoukhy, A. Karam, I. A. Shaban, and T. Zayed. 2021. “Modelling in off-site construction supply chain management: A review and future directions for sustainable modular integrated construction.” J. Cleaner Prod. 310 (Aug): 127503. https://doi.org/10.1016/j.jclepro.2021.127503.
Kim, K., and B. C. Paulson Jr. 2003. “Agent-based compensatory negotiation methodology to facilitate distributed coordination of project schedule changes.” J. Comput. Civ. Eng. 17 (1): 10–18. https://doi.org/10.1061/(ASCE)0887-3801(2003)17:1(10).
Konaka, A., D. W. Coit, and A. E. Smith. 2006. “Multi-objective optimization using genetic algorithms: A tutorial.” Reliab. Eng. Syst. Saf. 91 (9): 992–1007. https://doi.org/10.1016/j.ress.2005.11.018.
Kumanan, S., G. Jegan Jose, and K. Raja. 2006. “Multi-project scheduling using an heuristic and a genetic algorithm.” Int. J. Adv. Manuf. Technol. 31 (Nov): 360–366. https://doi.org/10.1007/s00170-005-0199-2.
Kwak, Y. H., and L. Ingall. 2007. “Exploring Monte Carlo simulation applications for project management.” Risk Manage. 9: 44–57.
Lawrence, S. R., and T. E. Morton. 1993. “Resource-constrained multi-project scheduling with tardy costs: Comparing myopic, bottleneck, and resource pricing heuristics.” Eur. J. Oper. Res. 64 (2): 168–187. https://doi.org/10.1016/0377-2217(93)90175-M.
Lee, D., and M. Kim. 2021. “Autonomous construction hoist system based on deep reinforcement learning in high-rise building construction.” Autom. Constr. 128 (Aug): 103737. https://doi.org/10.1016/j.autcon.2021.103737.
McCabe. 2003. “Monte Carlo simulation for schedule risks.” In Vol. 2 of Proc., 2003 Winter Simulation Conf., 1561–1565. New York: IEEE.
Metropolis, N., and S. Ulam. 1949. “The Monte Carlo method.” J. Am. Stat. Assoc. 44 (247): 335–341. https://doi.org/10.1080/01621459.1949.10483310.
Mohamed, Y., D. Borrego, L. Francisco, M. Al-Hussein, S. AbouRizk, and U. Hermann. 2007. “Simulation-based scheduling of module assembly yards: Case study.” Eng. Constr. Archit. Manage. 14 (3): 293–311. https://doi.org/10.1108/09699980710744926.
Mohanty, R. U., and M. K. Siddiq. 1989. “Multiple projects-multiple resources-constrained scheduling: Some studies.” Int. J. Prod. Res. 27 (2): 261–280. https://doi.org/10.1080/00207548908942546.
Moradi, H., and S. Shadrokh. 2019. “A robust scheduling for the multi-mode project scheduling problem with a given deadline under uncertainty of activity duration.” Int. J. Prod. Res. 57 (10): 3138–3167. https://doi.org/10.1080/00207543.2018.1552371.
Oguz, O., and H. Bala. 1994. “A comparative study of computational procedures for the resource constrained project scheduling problem.” Eur. J. Oper. Res. 72 (2): 406–416. https://doi.org/10.1016/0377-2217(94)90319-0.
Panahi, R., J. Louis, A. Podder, and C. Swanson. 2022. “Tracking volumetric units in modular factories for automated progress monitoring using computer vision.” In Proc., Construction Research Congress, 822–829. Reston, VA: ASCE.
Park, J., J. Chun, S. H. Kim, Y. Kim, and J. Park. 2021. “Learning to schedule job-shop problems: Representation and policy learning using graph neural network and reinforcement learning.” Int. J. Prod. Res. 59 (11): 3360–3377. https://doi.org/10.1080/00207543.2020.1870013.
Polat, G., D. Arditi, G. Ballard, and U. Mungen. 2006. “Economics of on-site vs. off-site fabrication of rebar.” Construct. Manage. Econ. 24 (11): 1185–1198. https://doi.org/10.1080/01446190500529432.
Powell, W. B. 2007. Approximate dynamic programming: Solving the curses of dimensionality. New York: Wiley.
Rahimi, I., A. H. Gandomi, K. Deb, F. Chen, and M. R. Nikoo. 2022. “Scheduling by NSGA-II: Review and bibliometric analysis.” Processes 10 (1): 98. https://doi.org/10.3390/pr10010098.
Ren, Z., G. Q. Shen, and X. L. Xue. 2013. “Failure caused by inappropriate construction methods: An expensive lesson.” J. Manage. Eng. 29 (1): 25–34. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000117.
Sacks, R., and M. Harel. 2006. “An economic game theory model of subcontractor resource allocation behavior.” Construct. Manage. Econ. 24 (8): 869–881. https://doi.org/10.1080/01446190600631856.
Sanchez, S. M., and T. W. Lucas. 2002. “Exploring the world of agent-based simulations: Simple models, complex analyses.” In Proc., Winter Simulation Conf., 116–126. New York: IEEE.
Shariatmadari, M., N. Nahavandi, S. H. Zegordi, and M. H. Sobhiyah. 2017. “Integrated resource management for simultaneous project selection and scheduling.” Comput. Ind. Eng. 109 (Jul): 39–47. https://doi.org/10.1016/j.cie.2017.04.003.
Si, T., H. X. Li, M. R. Hosseini, Y. Ji, and C. Liu. 2020. “A solution to Just-in-Time delivery for off-site construction: A conceptual model.” In Proc., Construction Research Congress 2020: Computer Applications, 345–353. Reston, VA: ASCE.
Soman, R. K., and M. Molina-Solana. 2022. “Automating look-ahead schedule generation for construction using linked-data based constraint checking and reinforcement learning.” Autom. Constr. 134 (Feb): 104069. https://doi.org/10.1016/j.autcon.2021.104069.
Sutton, R. S., and A. G. Barto. 2018. Reinforcement learning: An introduction. London: MIT Press.
Szepesvari, C. 2010. Algorithms for reinforcement learning. Kentfield, CA: Morgan & Claypool.
Taghaddos, H., U. Hermann, S. AbouRizk, and Y. Mohamed. 2014. “Simulation-based multiagent approach for scheduling modular construction.” J. Comput. Civ. Eng. 28 (2): 263–274. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000262.
Tao, L., Y. Dong, W. Chen, Y. Yang, L. Su, Q. Guo, and G. Wang. 2022. “A differential evolution with reinforcement learning for multi-objective assembly line feeding problem.” Comput. Ind. Eng. 174 (Dec): 108714. https://doi.org/10.1016/j.cie.2022.108714.
Toklu, Y. C. 2002. “Application of genetic algorithms to construction scheduling with or without resource constraints.” Can. J. Civ. Eng. 29 (3): 421–429. https://doi.org/10.1139/l02-034.
Tortorelli, A., M. Imran, F. Delli Priscoli, and F. Liberati. 2022. “A parallel deep reinforcement learning framework for controlling industrial assembly lines.” Electronics 11 (4): 539. https://doi.org/10.3390/electronics11040539.
Vercellis, C. 1994. “Constrained multi-project planning’s problems: A Lagrangean decomposition approach.” Eur. J. Oper. Res. 78 (2): 267–275. https://doi.org/10.1016/0377-2217(94)90389-1.
Wiley, V. D., R. F. Deckro, and J. A. Jackson Jr. 1998. “Optimization analysis for design and planning of multi-project programs.” Eur. J. Oper. Res. 107 (2): 492–506. https://doi.org/10.1016/S0377-2217(97)00334-2.
Williams, T. 2003. “The contribution of mathematical modelling to the practice of project management.” IMA J. Manage. Math. 14 (1): 3–30.
Yang, Z., and W. Lu. 2023. “Facility layout design for modular construction manufacturing: A comparison based on simulation and optimization.” Autom. Constr. 147 (Mar): 104713. https://doi.org/10.1016/j.autcon.2022.104713.
Yung, P., and B. Yip. 2010. “Construction quality in China during transition: A review of literature and empirical examination.” Int. J. Project Manage. 28 (1): 79–91. https://doi.org/10.1016/j.ijproman.2009.03.003.
Zhai, Y., R. Y. Zhong, and G. Q. Huang. 2018. “Buffer space hedging and coordination in prefabricated construction supply chain management.” Int. J. Prod. Econ. 200 (Jun): 192–206. https://doi.org/10.1016/j.ijpe.2018.03.014.
Zheng, D. X. M., S. T. Ng, and M. M. Kumaraswamy. 2004. “Applying a genetic algorithm-based multiobjective approach for time-cost optimization.” J. Constr. Eng. Manage. 130 (2): 168–176. https://doi.org/10.1061/(ASCE)0733-9364(2004)130:2(168).
Information & Authors
Information
Published In
Journal of Management in Engineering
Volume 40 • Issue 4 • July 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Sep 1, 2023
Accepted: Jan 19, 2024
Published online: Apr 16, 2024
Published in print: Jul 1, 2024
Discussion open until: Sep 16, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.