Chapter 3
Digital Twin–Enabled Health Monitoring of Construction Workers during Robotic Teleoperation
Publication: Digital Twins in Construction and the Built Environment
Abstract
Construction robots are increasingly being used on job sites to assist workers with repetitive and labor-intensive construction tasks. There is a lack of effective methods to evaluate workers' health and safety during human-robot interactions to take necessary preventive actions owing to the potential hazards to humans posed by construction robots. In this regard, digital twin (DT) technology holds promise for effectively characterizing the human-robot partnership and pertinent worker health information in real-time. In the context of human-robot interaction, DT technology can be effectively employed to simulate the interactions between workers and robots. This chapter proposes a framework for the construction of a DT-based platform that will support the health and safety monitoring of workers during human-robot teaming operations. It examines a virtual reality-based DT model for worker health monitoring using a combination of physiological signals, machine learning techniques, and immersive technologies.
Get full access to this chapter
View all available purchase options and get full access to this chapter.
Acknowledgments
The presented work was supported financially by a National Science Foundation Award (No. 2401745, “Future of Construction Workplace Health Monitoring”). Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation.
References
Ahn, C. R., S. Lee, C. Sun, H. Jebelli, et al. 2019. “Wearable sensing technology applications in construction safety and health.” J. Constr. Eng. Manage. 145 (11): 03119007.
Bangor, A., P. T. Kortum, and J. T. Miller. 2008. “An empirical evaluation of the system usability scale.” Int. J. Hum. Comput. Interact. 24 (6): 574–594.
Brooke, J. 1996. “SUS: A ‘quick and dirty’ usability scale.” In Usability evaluation in industry, edited by P. W. Jordan, B. Thomas, I. L. McClelland, and B. Weerdmeester. Boca Raton, FL: CRC Press.
Burghardt, A., D. Szybicki, P. Gierlak, K. Kurc, et al. 2020. “Programming of industrial robots using virtual reality and digital twins.” Appl. Sci. 10 (2): 486.
Busemeyer, J. R., and P. D. Bruza. 2012. Quantum models of cognition and decision. Cambridge: Cambridge University Press.
Cai, Y., Y. Wang, and M. Burnett. 2020. “Using augmented reality to build digital twin for reconfigurable additive manufacturing system.” J. Manuf. Syst. 56 (2020): 598–604.
Cheng, Y., L. Sun, C. Liu, and M. Tomizuka. 2020. “Towards efficient human–robot collaboration with robust plan recognition and trajectory prediction.” IEEE Robot. Autom. Lett. 5 (2): 2602–2609.
Dröder, K., P. Bobka, T. Germann, F. Gabriel, et al. 2018. “A machine learning-enhanced digital twin approach for human–robot-collaboration.” Procedia CIRP 76 (2018): 187–192.
Fukushima, Y., Y. Asai, S. Aoki, T. Yonezawa, et al. 2021. “DigiMobot: Digital twin for human–robot collaboration in indoor environments.” In Proc., IEEE Intelligent Vehicles Symp., 55–62. New York: Institute of Electrical and Electronics Engineers.
Garg, G., V. Kuts, and G. Anbarjafari. 2021. “Digital twin for FANUC robots: Industrial robot programming and simulation using virtual reality.” Sustainability 13 (18): 10336.
Haapalainen, E., S. Kim, J. F. Forlizzi, and A. K. Dey. 2010. “Psycho-physiological measures for assessing cognitive load.” In Proc., 12th ACM Int. Conf. on Ubiquitous Computing, 301–310. New York: ACM.
Hart, S. G., and L. E. Staveland. 1988. “Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research.” Adv. Psychol. 52 (1988): 139–183.
Havard, V., B. Jeanne, M. Lacomblez, and D. Baudry. 2019. “Digital twin and virtual reality: a co-simulation environment for design and assessment of industrial workstations.” Prod. Manuf. Res. 7 (1): 472–489.
Jebelli, H., S. Hwang, and S. Lee. 2018. “EEG-based workers’ stress recognition at construction sites.” Autom. Constr. 93 (2018): 315–324.
Kaarlela, T., S. Pieska, and T. Pitkaaho. 2020. “Digital twin and virtual reality for safety training.” In Proc., 11th IEEE Int. Conf. on Cognitive Infocommunications, 000115–000120. New York: IEEE.
Kanazawa, A., J. Kinugawa, and K. Kosuge. 2019. “Adaptive motion planning for a collaborative robot based on prediction uncertainty to enhance human safety and work efficiency.” IEEE Trans. Rob. 35 (4): 817–832.
Laaki, H., Y. Miche, and K. Tammi. 2019. “Prototyping a digital twin for real time remote control over mobile networks: Application of remote surgery.” IEEE Access 7: 20325–20336.
Liu, H., and L. Wang. 2020. “Remote human–robot collaboration: A cyber–physical system application for hazard manufacturing environment.” J. Manuf. Syst. 54 (2020): 24–34.
Liu, Y., and H. Jebelli. 2022. “Worker-aware robotic motion planner in construction for improved psychological well-being during worker–robot interaction.” In Proc., Construction Research Congress, 205–214. Reston, VA: ASCE.
Liu, Y., and H. Jebelli. 2023. “Intention-aware robot motion planning for safe worker–robot collaboration.” Comput.-Aided Civ. Infrastruct. Eng. 39 (15): 2242–2269.
Liu, Y., A. Ojha, S. Shayesteh, H. Jebelli, et al. 2023. “Human-centric robotic manipulation in construction: Generative adversarial networks based physiological computing mechanism to enable robots to perceive workers’ cognitive load.” Can. J. Civ. Eng. 50 (3): 224–238.
Maruyama, T., T. Ueshiba, M. Tada, H. Toda, et al. 2021. “Digital twin-driven human robot collaboration using a digital human.” Sensors 21 (24): 8266.
Meissner, A., A. Trübswetter, A. S. Conti-Kufner, and J. Schmidtler. 2021. “Friend or foe? Understanding assembly workers’ acceptance of human–robot collaboration.” ACM Trans. Hum. Robot. Interact. 10 (1): 1–30.
Nikolakis, N., V. Maratos, and S. Makris. 2019. “A cyber physical system (CPS) approach for safe human–robot collaboration in a shared workplace.” Robot. Comput. Integr. Manuf. 56 (2019): 233–243.
Ojha, A., M. Habibnezhad, H. Jebelli, and R. Leicht. 2022. “Barrier analysis of effective implementation of robotics in the construction industry.” In Proc., Construction Research Congress, 661–669. Reston, VA: ASCE.
Paas, F., J. E. Tuovinen, H. Tabbers, and P. W. Van Gerven. 2003. “Cognitive load measurement as a means to advance cognitive load theory.” Educ. Psychol. 38 (1): 63–71.
Pérez, L., S. Rodríguez-Jiménez, N. Rodríguez, R. Usamentiaga, et al. 2020. “Digital twin and virtual reality based methodology for multi-robot manufacturing cell commissioning.” Appl. Sci. 10 (10): 3633.
Ray, A., S. Phoha, and S. Sarkar. 2013. “Behavior prediction for decision & control in cognitive autonomous systems.” Eng. Mind Cognit. Sci. Rob. 9 (3): 263–271.
Sepasgozar, S. M. E. 2021. “Differentiating digital twin from digital shadow: Elucidating a paradigm shift to expedite a smart, sustainable built environment.” Buildings 11 (4): 151.
Shayesteh, S., and H. Jebelli. 2022. “Toward human-in-the-loop construction robotics: Understanding workers’ response through trust measurement during human–robot collaboration.” In Vol. 2-B of Construction Research Congress, 631–639. Reston, VA: ASCE.
Shayesteh, S., and H. Jebelli. 2023. “Investigating the impact of construction robots autonomy level on workers’ cognitive load.” In Proc., Canadian Society of Civil Engineering Annual Conf. 2021, 255–267. Singapore: Springer.
Shayesteh, S., and H. Jebelli. 2024. “Evaluating the feasibility of personalized health status feedback to enhance worker safety and well-being at construction job sites.” In Proc., Construction Research Congress. Reston, VA: ASCE.
Shayesteh, S., A. Ojha, Y. Liu, and H. Jebelli. 2023. “Human–robot teaming in construction: Evaluative safety training through the integration of immersive technologies and wearable physiological sensing.” Saf. Sci. 159 (2023): 106019.
Zacharaki, A., I. Kostavelis, A. Gasteratos, and I. Dokas. 2020. “Safety bounds in human robot interaction: A survey.” Saf. Sci. 127 (2020): 104667.
Zorzenon, R., F. L. Lizarelli, and D. B. A. de A. Moura. 2022. “What is the potential impact of industry 4.0 on health and safety at work?” Saf. Sci. 153 (2022): 105802.
Information & Authors
Information
Published In
Digital Twins in Construction and the Built Environment
Pages: 63 - 76
Editors: Houtan Jebelli, Ph.D., Somayeh Asadi, Ph.D., Ivan Mutis, Ph.D., Rui Liu, Ph.D., and Jack Cheng, Ph.D.
ISBN (Online): 978-0-7844-8560-6
Copyright
© 2024 by the American Society of Civil Engineers.
History
Published online: Sep 23, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.