Abstract
Augmented reality (AR) has become one of the most pervasive technologies for immersive applications with potential to improve worker performance in construction. However, the cognitive impact of using AR is still being studied and a comprehensive assessment of changes on user’s cognitive status is needed. This study proposed an electroencephalogram (EEG)-based mental workload evaluation method for a wood frame assembly, comparing the differences between two information display types (conformal model and paper layout). Additionally, a task performance analysis was conducted for completion rate and accuracy. The experiment recruited thirty construction students with similar backgrounds who were novice AR users. The findings indicated that AR improved the assembling efficiency with significantly higher completion () in comparison to the paper display. Furthermore, both mean and maximum values during the task process were analyzed to compare the difference between the displays, where AR display generated significantly less mental work in mean () and maximal () values. The comparison of the changing trends of mental workload increment in assembly revealed that AR display maintained lower mental workload and smaller fluctuation (standard deviation) during the entire assembly. The outcomes examined the usability of AR head-mounted display (HMD) in the construction assembly task with a moment-to-moment physiological evaluation of user’s mental workload. This work shows that EEG-based mental workload evaluation can be used as an effective method for revealing changes in workers’ mental status in real time during assembly, which has implications for future research in areas such as hazard recognition and situational awareness evaluation.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code generated or used during the study are proprietary or confidential in nature and may only be provided with restrictions.
References
Abdelhamid, T. S., and J. G. Everett. 2002. “Physiological demands during construction work.” J. Constr. Eng. Manage. 128 (5): 427–437. https://doi.org/10.1061/(ASCE)0733-9364(2002)128:5(427).
Alves, J., B. Marques, M. Oliveira, T. Araújo, P. Dias, and B. S. Santos. 2019. “Comparing spatial and mobile augmented reality for guiding assembling procedures with task validation.” In Proc., 2019 IEEE Int. Conf. on Autonomous Robot Systems and Competitions (ICARSC), 1–6. New York: IEEE.
Aromaa, S., A. Väätänen, I. Aaltonen, V. Goriachev, K. Helin, and J. Karjalainen. 2020. “Awareness of the real-world environment when using augmented reality head-mounted display.” Appl. Ergon. 88 (Oct): 103145. https://doi.org/10.1016/j.apergo.2020.103145.
Bosch, T., R. Könemann, H. de Cock, and G. Van Rhijn. 2017. “The effects of projected versus display instructions on productivity, quality and workload in a simulated assembly task.” In Proc., 10th Int. Conf. on PErvasive Technologies Related to Assistive Environments, 412–415. New York: Association for Computing Machinery.
Chalhoub, J., and S. K. Ayer. 2019. “Exploring the performance of an augmented reality application for construction layout tasks.” Multimedia Tools Appl. 78 (24): 35075–35098. https://doi.org/10.1007/s11042-019-08063-5.
Chen, J., X. Song, and Z. Lin. 2016. “Revealing the ‘Invisible Gorilla’ in construction: Estimating construction safety through mental workload assessment.” Autom. Constr. 63 (Mar): 173–183. https://doi.org/10.1016/j.autcon.2015.12.018.
Dadi, G. B., P. M. Goodrum, T. R. Taylor, and C. M. Carswell. 2014. “Cognitive workload demands using 2D and 3D spatial engineering information formats.” J. Constr. Eng. Manage. 140 (5): 04014001. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000827.
Delorme, A., and S. Makeig. 2004. “EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis.” J. Neurosci. Methods 134 (1): 9–21. https://doi.org/10.1016/j.jneumeth.2003.10.009.
Deshpande, A., and I. Kim. 2018. “The effects of augmented reality on improving spatial problem solving for object assembly.” Adv. Eng. Inf. 38 (Oct): 760–775. https://doi.org/10.1016/j.aei.2018.10.004.
Dimitriadis, S. I., Y. Sun, K. Kwok, N. A. Laskaris, N. Thakor, and A. Bezerianos. 2015. “Cognitive workload assessment based on the tensorial treatment of EEG estimates of cross-frequency phase interactions.” Ann. Biomed. Eng. 43 (4): 977–989. https://doi.org/10.1007/s10439-014-1143-0.
Drascic, D., and P. Milgram. 1996. “Perceptual issues in augmented reality.” In Vol. 2653 of Proc., Stereoscopic Displays and Virtual Reality Systems III, 123–124. Cergy-Pontoise, France: SPIE. https://doi.org/10.1117/12.237425.
Funke, G., B. Knott, V. F. Mancuso, A. Strang, J. Estepp, L. Menke, R. Brown, A. Dukes, and B. Miller. 2013. “Evaluation of subjective and EEG-based measures of mental workload.” In Proc., HCI Int. 2013-Posters’ Extended Abstracts: Int. Conf., edited by C. Stephanidis, 412–416. Berlin: Springer.
Gevins, A., M. E. Smith, L. McEvoy, and D. Yu. 1997. “High-resolution EEG mapping of cortical activation related to working memory: Effects of task difficulty, type of processing, and practice.” Cereb. Cortex 7 (4): 374–385. https://doi.org/10.1093/cercor/7.4.374.
Harmony, T. 2013. “The functional significance of delta oscillations in cognitive processing.” Front. Integr. Neurosci. 7 (Dec): 83. https://doi.org/10.3389/fnint.2013.00083.
Hart, S. G., and L. E. Staveland. 1988. “Development of NASA-TLX (task load index): Results of empirical and theoretical research.” In Advances in psychology, edited by P. A. Hancock and N. Meshkati, 139–183. Amsterdam, Netherlands: North-Holland.
Hou, L., X. Wang, L. Bernold, and P. E. Love. 2013a. “Using animated augmented reality to cognitively guide assembly.” J. Comput. Civ. Eng. 27 (5): 439–451. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000184.
Hou, L., X. Wang, and M. Truijens. 2013b. “Using augmented reality to facilitate piping assembly: An experiment-based evaluation.” J. Comput. Civ. Eng. 29 (1): 05014007. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000344.
Hwang, S., H. Jebelli, B. Choi, M. Choi, and S. Lee. 2018. “Measuring workers’ emotional state during construction tasks using wearable EEG.” J. Constr. Eng. Manage. 144 (7): 04018050. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001506.
Jebelli, H., S. Hwang, and S. Lee. 2017. “EEG signal-processing framework to obtain high-quality brain waves from an off-the-shelf wearable EEG device.” J. Comput. Civ. Eng. 32 (1): 04017070. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000719.
Jebelli, H., S. Hwang, and S. Lee. 2018. “EEG-based workers’ stress recognition at construction sites.” Autom. Constr. 93 (Sep): 315–324. https://doi.org/10.1016/j.autcon.2018.05.027.
Jeffri, N. F. S., and D. R. A. Rambli. 2021. “A review of augmented reality systems and their effects on mental workload and task performance.” Heliyon 7 (3): e06277. https://doi.org/10.1016/j.heliyon.2021.e06277.
Jung, T.-P., S. Makeig, C. Humphries, T.-W. Lee, M. J. Mckeown, V. Iragui, and T. J. Sejnowski. 2000. “Removing electroencephalographic artifacts by blind source separation.” Psychophysiology 37 (2): 163–178. https://doi.org/10.1111/1469-8986.3720163.
Keil, J., A. Korte, A. Ratmer, D. Edler, and F. Dickmann. 2020. “Augmented reality (AR) and spatial cognition: Effects of holographic grids on distance estimation and location memory in a 3D indoor scenario.” J. Photogramm. Remote Sens. Geoinf. Sci. 88 (2): 165–172. https://doi.org/10.1007/s41064-020-00104-1.
Kim, J., and J. Irizarry. 2021. “Evaluating the use of augmented reality technology to improve construction management student’s spatial skills.” Int. J. Construct. Educ. Res. 17 (2): 99–116. https://doi.org/10.1080/15578771.2020.1717680.
Kim, K.-H., and K.-Y. Wohn. 2011. “Effects on productivity and safety of map and augmented reality navigation paradigms.” IEICE Trans. Inf. Syst. 94 (5): 1051–1061. https://doi.org/10.1587/transinf.E94.D.1051.
Koessler, L., L. Maillard, A. Benhadid, J.-P. Vignal, M. Braun, and H. Vespignani. 2007. “Spatial localization of EEG electrodes.” Neurophysiologie Clin./Clin. Neurophysiol. 37 (2): 97–102. https://doi.org/10.1016/j.neucli.2007.03.002.
Kwiatek, C., M. Sharif, S. Li, C. Haas, and S. Walbridge. 2019. “Impact of augmented reality and spatial cognition on assembly in construction.” Autom. Constr. 108 (Dec): 102935. https://doi.org/10.1016/j.autcon.2019.102935.
Lee, T.-W., M. Girolami, and T. J. Sejnowski. 1999. “Independent component analysis using an extended infomax algorithm for mixed subgaussian and supergaussian sources.” Neural Comput. 11 (2): 417–441. https://doi.org/10.1162/089976699300016719.
Loch, F., F. Quint, and I. Brishtel. 2016. “Comparing video and augmented reality assistance in manual assembly.” In Proc., 2016 12th Int. Conf. on Intelligent Environments, 147–150. New York: IEEE.
Pion-Tonachini, L., S. Makeig, and K. Kreutz-Delgado. 2017. “Crowd labeling latent Dirichlet allocation.” Knowl. Inf. Syst. 53 (3): 749–765. https://doi.org/10.1007/s10115-017-1053-1.
Qin, Y., E. Bloomquist, T. Bulbul, J. Gabbard, and K. Tanous. 2021. “Impact of information display on worker performance for wood frame wall assembly using AR HMD under different task conditions.” Adv. Eng. Inf. 50 (Oct): 101423. https://doi.org/10.1016/j.aei.2021.101423.
Qin, Y., and T. Bulbul. 2020. “Towards an EEG based mental workload evaluation method for construction workers’ HMD AR use.” In Proc., Construction Research Congress 2020: Computer Applications. Reston, VA: ASCE. https://doi.org/10.1061/9780784482865.070.
Reddy, A. G., and S. Narava. 2013. “Artifact removal from EEG signals.” Int. J. Comput. Appl. 77 (13): 17–19. https://doi.org/10.5120/13543-1175.
Soria, C., and M. Roth. 2018. “Unreal reality: An empirical investigation of augmented reality effects on spatial cognition in landscape architecture.” J. Digital Landscape Archit. 3 (Mar): 150–162. https://doi.org/10.14627/537642016.
Stadler, S., K. Kain, M. Giuliani, N. Mirnig, G. Stollnberger, and M. Tscheligi. 2016. “Augmented reality for industrial robot programmers: Workload analysis for task-based, augmented reality-supported robot control.” In Proc., 2016 25th IEEE Int. Symp. on Robot and Human Interactive Communication (RO-MAN), 179–184. New York: IEEE.
Stoica, P., and R. L. Moses. 1997. Introduction to spectral analysis. London: Pearson Education.
Sweller, J. 1988. “Cognitive load during problem solving: Effects on learning.” Cognit. Sci. 12 (2): 257–285. https://doi.org/10.1207/s15516709cog1202_4.
Tang, A., C. Owen, F. Biocca, and W. Mou. 2004. “Performance evaluation of augmented reality for directed assembly.” In Virtual and augmented reality applications in manufacturing, 311–331. London: Springer.
Teplan, M. 2002. “Fundamentals of EEG measurement.” Meas. Sci. Rev. 2 (2): 1–11.
Vidulich, M. A., and P. S. Tsang. 2012. “Mental workload and situation awareness.” In Handbook of human factors and ergonomics, 243–273. Hoboken, NJ: Wiley.
Wang, D., H. Li, and J. Chen. 2019. “Detecting and measuring construction workers’ vigilance through hybrid kinematic-EEG signals.” Autom. Constr. 100 (Apr): 11–23. https://doi.org/10.1016/j.autcon.2018.12.018.
Wang, X., and P. S. Dunston. 2006. “Compatibility issues in Augmented Reality systems for AEC: An experimental prototype study.” Autom. Constr. 15 (3): 314–326. https://doi.org/10.1016/j.autcon.2005.06.002.
Wilschut, E. S., R. Könemann, M. S. Murphy, G. J. Van Rhijn, and T. Bosch. 2019. “Evaluating learning approaches for product assembly: Using chunking of instructions, spatial augmented reality and display based work instructions.” In Proc., 12th ACM Int. Conf. on PErvasive Technologies Related to Assistive Environments, 376–381. New York: Association for Computing Machinery.
Yang, Z., J. Shi, W. Jiang, Y. Sui, Y. Wu, S. Ma, C. Kang, and H. Li. 2019. “Influences of augmented reality assistance on performance and cognitive loads in different stages of assembly task.” Front. Psychol. 10 (Jul): 1703. https://doi.org/10.3389/fpsyg.2019.01703.
Zhao, C., C. Zheng, M. Zhao, Y. Tu, and J. Liu. 2011. “Multivariate autoregressive models and kernel learning algorithms for classifying driving mental fatigue based on electroencephalographic.” Expert Syst. Appl. 38 (3): 1859–1865. https://doi.org/10.1016/j.eswa.2010.07.115.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 149 • Issue 9 • September 2023
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Dec 9, 2022
Accepted: May 4, 2023
Published online: Jul 12, 2023
Published in print: Sep 1, 2023
Discussion open until: Dec 12, 2023
ASCE Technical Topics:
- Business management
- Comparative studies
- Construction engineering
- Construction management
- Disaster risk management
- Disasters and hazards
- Education
- Employment
- Engineering fundamentals
- Frames
- Labor
- Methodology (by type)
- Models (by type)
- Personnel management
- Practice and Profession
- Research methods (by type)
- Simulation models
- Structural engineering
- Structural members
- Structural systems
- Students
- Wood frames
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Jeonghyeun Chae, Sungjoo Hwang, Youngcheol Kang, Measuring Habituation to Auditory Warnings Using Behavioral and Physiological Data, Journal of Construction Engineering and Management, 10.1061/JCEMD4.COENG-14450, 150, 7, (2024).
- Florence Y. Y. Ling, Gerald Tze Hon Heng, Alice Chang-Richards, Xichen Chen, Tak Wing Yiu, Impact of Digital Technology Adoption on the Comparative Advantage of Architectural, Engineering, and Construction Firms in Singapore, Journal of Construction Engineering and Management, 10.1061/JCEMD4.COENG-13743, 149, 12, (2023).