A Dualistic Perspective of Opportunity and Risk: The Impact of Head-Mounted Augmented Reality on Construction Onsite Hazard Identification of Workers
Publication: Journal of Construction Engineering and Management
Volume 150, Issue 11
Abstract
Head-mounted augmented reality (HMD AR) technology in construction has shown promise in enhancing operational efficiency. Concurrently, the critical role of hazard identification (HI) in ensuring worker safety is well-documented. Despite this, studies delving into HMD AR’s impact on HI, especially across varying task complexities, are limited. This study aims to bridge this gap by examining the influence of HMD AR on construction workers’ HI in diverse task environments. The research engaged 48 participants in an experiment to evaluate their hazard situation awareness (HSA), cognitive load (CL), and task performance (TP). These were assessed using the Situation Awareness Global Assessment Technique (SAGAT), the Situation Awareness Rating Technique (SART), the National Aeronautics and Space Administration Task Load Index (NASA-TLX), and measures of task accuracy. Findings indicated that environmental conditions significantly modulate HMD AR’s effect on HI. In simple and safe scenes, HMD AR has a negligible impact on HI and might enhance hazard detection capabilities. However, in complex and hazardous situations with intense workloads, HMD AR can impair HI, potentially increasing safety risks. In other words, whether HMD AR brings opportunities or risks for HI depends on the different environmental conditions. This research contributes to the nascent field of cognitive safety in HMD AR, specifically regarding HI in construction, augmenting the current understanding and laying a theoretical groundwork for HMD AR’s future safety applications in the architecture, engineering, and construction (AEC) industry. The insights gained inform the prudent implementation, training, and interface design of HMD AR on construction sites. In the context of HMD AR’s rapid evolution, this study underscores the importance of critically assessing safety implications to guide the responsible utilization of new technologies.
Practical Applications
The findings from this study reveal that the impact of HMD AR on HI is contingent upon the complexity and safety of the work environment. In scenes that are straightforward and safe, HMD AR has been shown to improve HI among participants. Conversely, in environments that are complex and fraught with danger, HMD AR may hinder HI, increasing the likelihood of hazards. The conclusions of this study suggest that the safety management and application of HMD AR on construction sites should consider the opportunities and risks of using HMD AR in different environmental conditions. It is advised that decision makers on construction sites evaluate the suitability of HMD AR deployment based on the specific complexities of their scenes. In instances where the potential hazards are intricate, the use of HMD AR might be inadvisable. For sites already utilizing HMD AR, it is crucial to provide additional safety training to users. Such training should aim to enhance awareness of how HMD AR might affect HI, equipping workers with the knowledge to mitigate potential risks effectively. Furthermore, developers of HMD AR technologies are encouraged to focus on improving the safety features of their products. A key area of improvement is the optimization of information density on the HMD AR display interface. By doing so, developers can help prevent cognitive overload among users, thereby minimizing any adverse effects on HI. This approach not only enhances the safety on construction sites but also ensures that the benefits of HMD AR can be fully realized without compromising worker safety.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or codes that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
The authors thank the funding of the Key Project of Science and Technology Innovation Plan of Shanghai Science and Technology Commission (23692109200) and insightful suggestions from the anonymous reviewers and the editors. Authors Jiale Liu and Xiaoli Yan contributed equally as the first authors of this article.
References
Abbas, A. 2021. “Understanding human factors issues in immersive virtual and augmented reality applications for construction.” Ph.D. thesis, Dept. of Building and Real Estate, Hong Kong Polytechnic Univ.
Abbas, A., J. Seo, and M. Kim. 2020. “Impact of mobile augmented reality system on cognitive behavior and performance during rebar inspection tasks.” J. Comput. Civ. Eng. 34 (6): 04020050. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000931.
Albert, A., M. R. Hallowell, B. Kleiner, A. Chen, and M. Golparvar-Fard. 2014. “Enhancing construction hazard recognition with high-fidelity augmented virtuality.” J. Constr. Eng. Manage. 140 (7): 04014024. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000860.
Alizadehsalehi, S., A. Hadavi, and J. C. Huang. 2020. “From BIM to extended reality in AEC industry.” Autom. Constr. 116 (Aug): 103254. https://doi.org/10.1016/j.autcon.2020.103254.
Alonso, F. M., R. Kajastila, T. Takala, M. Matveinen, M. Kytö, and P. Hämäläinen. 2016. “Virtual ball catching performance in different camera views.” In Proc., 20th Int. Academic Mindtrek Conf., 104–112. New York: Association for Computing Machinery.
Ariansyah, D., J. A. Erkoyuncu, I. Eimontaite, T. Johnson, A. M. Oostveen, S. Fletcher, and S. Sharples. 2022. “A head mounted augmented reality design practice for maintenance assembly: Toward meeting perceptual and cognitive needs of AR users.” Appl. Ergon. 98 (Jan): 103597. https://doi.org/10.1016/j.apergo.2021.103597.
Aromaa, S., A. Vaatanen, I. Aaltonen, V. Goriachev, K. Helin, and J. Karjalainen. 2020. “Awareness of the real-world environment when using augmented reality head-mounted display.” Appl. Ergon. 88 (Oct): 103145. https://doi.org/10.1016/j.apergo.2020.103145.
Baumeister, J., S. Y. Ssin, N. A. ElSayed, J. Dorrian, D. P. Webb, J. A. Walsh, T. M. Simon, A. Irlitti, R. T. Smith, and M. Kohler. 2017. “Cognitive cost of using augmented reality displays.” IEEE Trans. Vis. Comput. Graphics 23 (11): 2378–2388. https://doi.org/10.1109/TVCG.2017.2735098.
Bhandari, N., and E. O’Neill. 2020. “Influence of perspective on dynamic tasks in virtual reality.” In Proc., 2020 IEEE Conf. on Virtual Reality and 3D User Interfaces (VR), 939–948. New York: IEEE.
Bhandari, S., M. R. Hallowell, L. V. Boven, K. M. Welker, M. Golparvar-Fard, and J. Gruber. 2020. “Using augmented virtuality to examine how emotions influence construction-hazard identification, risk assessment, and safety decisions.” J. Constr. Eng. Manage. 146 (2): 04019102. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001755.
Chen, J., J. E. Taylor, and S. Comu. 2017. “Assessing task mental workload in construction projects: A novel electroencephalography approach.” J. Constr. Eng. Manage. 143 (8): 04017053. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001345.
Choi, S., and J. S. Park. 2021. “Development of augmented reality system for productivity enhancement in offshore plant construction.” J. Mar. Sci. Eng. 9 (2): 209. https://doi.org/10.3390/jmse9020209.
Chu, M., J. Matthews, and P. E. D. Love. 2018. “Integrating mobile building information modelling and augmented reality systems: An experimental study.” Autom. Constr. 85 (Jan): 305–316. https://doi.org/10.1016/j.autcon.2017.10.032.
Delgado, J. M. D., L. Oyedele, P. Demian, and T. Beach. 2020. “A research agenda for augmented and virtual reality in architecture, engineering and construction.” Adv. Eng. Inf. 45 (Aug): 101122. https://doi.org/10.1016/j.aei.2020.101122.
Deng, S. W., R. Peng, and Y. G. Pan. 2022. “A cognitive failure model of construction workers’ unsafe behavior.” Adv. Civ. Eng. 2022 (1): 2576600. https://doi.org/10.1155/2022/2576600.
Denisova, A., and P. Cairns. 2015. “First person vs. third person perspective in digital games.” In Proc., 33rd Annual ACM Conf. on Human Factors in Computing Systems, 145–148. New York: Association for Computing Machinery.
Doswell, J. T., and A. Skinner. 2014. “Augmenting human cognition with adaptive augmented reality.” In Proc., Foundations of Augmented Cognition. Advancing Human Performance and Decision-Making through Adaptive Systems: 8th Int. Conf., AC 2014, Held as Part of HCI Int. 2014, Heraklion, Crete, Greece, June 22-27, 2014, 104–113. Cham, Switzerland: Springer.
Eiris, R., E. Jain, M. Gheisari, and A. Wehle. 2021. “Online hazard recognition training: Comparative case study of static images, cinemagraphs, and videos.” J. Constr. Eng. Manage. 147 (8): 04021082. https://doi.org/10.1061/(ASCE)CO.1943-7862.0002090.
Endsley, M. R., S. J. Selcon, T. D. Hardiman, and D. G. Croft. 1998. “Comparative analysis of SAGAT and SART for evaluations of situation awareness.” Proc. Hum. Factors Ergon. Soc. 42 (1): 82–86. https://doi.org/10.1177/154193129804200119.
Gao, W., X. Yan, and T. Li. 2024. “Influencing factors and critical paths of virtual reality adoption by AEC enterprises based on TOE-SNA.” J. Constr. Eng. Manage. 150 (2): 04023162. https://doi.org/10.1061/JCEMD4.COENG-13242.
Guo, H.-J., J. Z. Bakdash, L. R. Marusich, and B. Prabhakaran. 2022. “Augmented reality and mixed reality measurement under different environments: A survey on head-mounted devices.” IEEE Trans. Instrum. Meas. 71 (Oct): 1–15. https://doi.org/10.1109/TIM.2022.3218303.
Han, Y., Z. Yin, J. Liu, R. Jin, K. Gidado, N. Painting, Y. Yang, and L. Yan. 2019. “Defining and testing a safety cognition framework incorporating safety hazard perception.” J. Constr. Eng. Manage. 145 (12): 04019081. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001718.
Han, Y., Z. Yin, J. Zhang, R. Jin, and T. Yang. 2020. “Eye-tracking experimental study investigating the influence factors of construction safety hazard recognition.” J. Constr. Eng. Manage. 146 (8): 04020091. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001884.
Hart, S. G. 2006. “NASA-task load index (NASA-TLX); 20 years later.” In Proc., Human Factors and Ergonomics Society Annual Meeting, 904–908. Los Angeles: SAGE.
Hasanzadeh, S., B. Esmaeili, and M. D. Dodd. 2018. “Examining the relationship between construction workers’ visual attention and situation awareness under fall and tripping hazard conditions: Using mobile eye tracking.” J. Constr. Eng. Manage. 144 (7): 04018060. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001516.
Hu, M., T. Shealy, M. Hallowell, and D. Hardison. 2018. “Advancing construction hazard recognition through neuroscience: Measuring cognitive response to hazards using functional near infrared spectroscopy.” In Proc., Construction Research Congress, 134–143. Reston, VA: ASCE.
Hu, Z., W. T. Chan, H. Hu, and F. Xu. 2023. “Cognitive factors underlying unsafe behaviors of construction workers as a tool in safety management: A review.” J. Constr. Eng. Manage. 149 (3): 03123001. https://doi.org/10.1061/JCEMD4.COENG-11820.
Jazayeri, E., and G. B. Dadi. 2020. “Hazard recognition and risk perception skills among union electricians.” J. Constr. Eng. Manage. 146 (9): 04020108. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001874.
Jeffri, N. F. S., and D. R. A. Rambli. 2021. “A review of augmented reality systems and their effects on mental workload and task performance.” Heliyon 7 (3): e06277. https://doi.org/10.1016/j.heliyon.2021.e06277.
Jung, H., W. Seo, and Y. Kang. 2023. “Differences in workers’ safety behavior by project size and risk level of work in South Korea.” J. Constr. Eng. Manage. 149 (5): 04023028. https://doi.org/10.1061/JCEMD4.COENG-12890.
Kang, H., J. Ko, H. Park, and H. Hong. 2019. “Effect of outside view on attentiveness in using see-through type augmented reality device.” Displays 57 (Apr): 1–6. https://doi.org/10.1016/j.displa.2019.02.001.
Kim, K., H. Kim, and H. Kim. 2017. “Image-based construction hazard avoidance system using augmented reality in wearable device.” Autom. Constr. 83 (Nov): 390–403. https://doi.org/10.1016/j.autcon.2017.06.014.
Kim, S., H. Lee, S. Hwang, J.-S. Yi, and J. Son. 2021. “Construction workers’ awareness of safety information depending on physical and mental load.” J. Asian Archit. Build. Eng. 21 (3): 1067–1077. https://doi.org/10.1080/13467581.2021.1908899.
Kolaei, A. Z., E. Hedayati, M. Khanzadi, and G. G. Amiri. 2022. “Challenges and opportunities of augmented reality during the construction phase.” Autom. Constr. 143 (Nov): 104586.
Liao, P.-C., X. Sun, and D. Zhang. 2021. “A multimodal study to measure the cognitive demands of hazard recognition in construction workplaces.” Saf. Sci. 133 (Jan): 105010. https://doi.org/10.1016/j.ssci.2020.105010.
Lyell, D., F. Magrabi, and E. Coiera. 2018. “The effect of cognitive load and task complexity on automation bias in electronic prescribing.” Hum. Factors 60 (7): 1008–1021. https://doi.org/10.1177/0018720818781224.
MacFarland, T. W., and J. M. Yates. 2016. “Mann–Whitney U test.” In Introduction to nonparametric statistics for the biological sciences using R, edited by T. W. MacFarland and J. M. Yates, 103–132. Cham: Springer International Publishing.
MHURDPRC (Ministry of Housing and Urban-Rural Development of the People’s Republic of China). 2020. “General Office of the Ministry of Housing and Urban-Rural Development on 2020 notification of production safety accidents of housing municipal works.” Accessed September 1, 2023. https://www.mohurd.gov.cn/gongkai/zhengce/zhengcefilelib/202210/20221026_768565.html.
Mitropoulos, P., and B. Memarian. 2013. “Task demands in masonry work: Sources, performance implications, and management strategies.” J. Constr. Eng. Manage. 139 (5): 581–590. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000586.
Moreira, L. S., R. C. Ruschel, and A. H. Behzadan. 2018. “Building owner manual assisted by augmented reality: A case from Brazil.” In Proc., Construction Research Congress, 460–469. Reston, VA: ASCE.
Noghabaei, M., K. Han, and A. Albert. 2021. “Feasibility study to identify brain activity and eye-tracking features for assessing hazard recognition using consumer-grade wearables in an immersive virtual environment.” J. Constr. Eng. Manage. 147 (9): 04021104. https://doi.org/10.1061/(ASCE)CO.1943-7862.0002130.
Pooladvand, S., and S. Hasanzadeh. 2023. “Impacts of stress on workers’ risk-taking behaviors: Cognitive tunneling and impaired selective attention.” J. Constr. Eng. Manage. 149 (8): 04023060. https://doi.org/10.1061/JCEMD4.COENG-13339.
Qin, Y., E. Bloomquist, T. Bulbul, J. Gabbard, and K. Tanous. 2021. “Impact of information display on worker performance for wood frame wall assembly using AR HMD under different task conditions.” Adv. Eng. Inf. 50 (Oct): 101423. https://doi.org/10.1016/j.aei.2021.101423.
Qin, Y., and T. Bulbul. 2020. “Towards an EEG based mental workload evaluation method for construction workers’ HMD AR use.” In Proc., Construction Research Congress 2020, 659–667. Reston, VA: ASCE.
Qin, Y., and T. Bulbul. 2023. “An EEG-based mental workload evaluation for AR head-mounted display use in construction assembly tasks.” J. Constr. Eng. Manage. 149 (9): 04023088. https://doi.org/10.1061/JCEMD4.COENG-13438.
Rapp, A., and C. Gena. 2014. “Immersion and Involvement in a 3D Training Environment: Experimenting different points of view.” Proc., IEEE Int. Conf. on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA), 18–23. New York: IEEE.
Sabeti, S., O. Shoghli, M. Baharani, and H. Tabkhi. 2021. “Toward AI-enabled augmented reality to enhance the safety of highway work zones: Feasibility, requirements, and challenges.” Adv. Eng. Inf. 50 (Oct): 101429. https://doi.org/10.1016/j.aei.2021.101429.
Standardization Administration of China. 1993. Safety code of power supply and consumption for installation construction engineering. GB 50194-93. Beijing: China Planning Press.
Tan, Y., W. Xu, S. Li, and K. Chen. 2022. “Augmented and virtual reality (AR/VR) for education and training in the AEC industry: A systematic review of research and applications.” Buildings 12 (10): 1529. https://doi.org/10.3390/buildings12101529.
Taylor, R. 1990. “Situation awareness rating technique: The development of a tool for aircrew system design.” In Proc., AGARD Conf. Proc. Oxfordshire, UK: Routledge.
Taylor, S., and P. A. Todd. 1995. “Understanding information technology usage: A test of competing models.” Inf. Syst. Res. 6 (2): 144–176. https://doi.org/10.1287/isre.6.2.144.
Tixier, A. J. P., M. R. Hallowell, A. Albert, L. van Boven, and B. M. Kleiner. 2014. “Psychological antecedents of risk-taking behavior in construction.” J. Constr. Eng. Manage. 140 (11): 04014052. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000894.
Tixier, A. J.-P. 2013. “Psychological antecedents of risk-taking behavior in construction.” J. Constr. Eng. Manage. 140 (11): 04014052. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000894.
Umair, M., A. Sharafat, D.-E. Lee, and J. Seo. 2022. “Impact of virtual reality-based design review system on user’s performance and cognitive behavior for building design review tasks.” Appl. Sci. 12 (14): 7249. https://doi.org/10.3390/app12147249.
Urban, H., G. Pelikan, and C. Schranz. 2022. “Augmented reality in AEC education: A case study.” Buildings 12 (4): 391. https://doi.org/10.3390/buildings12040391.
US Department of Labor. 2022. “Census of fatal occupational injuries summary.” Accessed September 23, 2023. https://www.bls.gov/news.release/cfoi.nr0.htm.
Wenk, N., J. Penalver-Andres, R. Palma, K. A. Buetler, R. Muri, T. Nef, and L. Marchal-Crespo. 2019. “Reaching in several realities: Motor and cognitive benefits of different visualization technologies.” In Proc., 16th IEEE Int. Conf. on Rehabilitation Robotics (ICORR), 1037–1042. New York: IEEE.
Yang, A., M. Z. Han, Q. C. Zeng, and Y. H. Sun. 2021. “Adopting building information modeling (BIM) for the development of smart buildings: A review of enabling applications and challenges.” Adv. Civ. Eng. 2021 (1): 8811476. https://doi.org/10.1155/2021/8811476.
Yu, W.-D., K.-C. Wang, and H.-T. Wu. 2022. “Empirical comparison of learning effectiveness of immersive virtual reality-based safety training for novice and experienced construction workers.” J. Constr. Eng. Manage. 148 (9): 04022078. https://doi.org/10.1061/(ASCE)CO.1943-7862.0002337.
Zhang, P., N. Li, Z. Jiang, D. Fang, and C. J. Anumba. 2019. “An agent-based modeling approach for understanding the effect of worker-management interactions on construction workers’ safety-related behaviors.” Autom. Constr. 97 (Jan): 29–43. https://doi.org/10.1016/j.autcon.2018.10.015.
Zhao, X., M. Zhang, X. Fan, Z. Sun, M. Li, W. Li, and L. Huang. 2023. “Extended reality for safe and effective construction management: State-of-the-art, challenges, and future directions.” Buildings 13 (1): 155. https://doi.org/10.3390/buildings13010155.
Zhu, Q., J. Du, Y. Shi, and P. Wei. 2021. “Neurobehavioral assessment of force feedback simulation in industrial robotic teleoperation.” Autom. Constr. 126 (Jun): 103674. https://doi.org/10.1016/j.autcon.2021.103674.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 150 • Issue 11 • November 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Oct 26, 2023
Accepted: Jun 6, 2024
Published online: Aug 29, 2024
Published in print: Nov 1, 2024
Discussion open until: Jan 29, 2025
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.