Analyzing Trust Dynamics in Human–Robot Collaboration through Psychophysiological Responses in an Immersive Virtual Construction Environment
Publication: Journal of Computing in Civil Engineering
Volume 38, Issue 4
Abstract
Human–robot collaboration (HRC) has emerged as a promising frontier within the construction industry, offering the potential to enhance productivity, safety, and efficiency. The effectiveness of HRC critically depends on the degree of trust that workers place in their robots, and establishing a seamless level of trust in robots is essential to realize the full benefits of HRC. Despite the extensive exploration of trust dynamics in various industries, there is a notable research gap with regard to trust in construction robots, which possess distinctive characteristics in terms of appearance, capabilities, and interaction compared to robots in other sectors. Therefore, in this study, we analyzed trust dynamics within the context of HRC during construction tasks. Both subjective survey data and objective psychophysiological data—including heart rate variability (HRV), electrodermal activity (EDA), and electroencephalogram (EEG)-based emotional valence and arousal—were employed as human trust measures. We conducted experiments for bricklaying tasks in an immersive virtual construction environment and analyzed multifaceted robot factors—including workspace environment, level of interaction, and robot speed, proximity, and angle of approach—and their relationships with human trust measures using statistical analysis, such as -test, two-way ANOVA, Spearman’s rank correlation, and moderation analysis. The results indicated that workspace environment and level of interaction were the most significant robot factors affecting human trust. EDA exhibited the most sensitivity to variations in robot factors. It was also observed that the effect of speed, proximity, and angle of approach were also dependent on level of interaction and type of workspace environment. There was a significant positive correlation between proximity and perceived trust. The findings of this study contribute to the optimization of robot design and interaction protocols for construction tasks, fostering greater worker trust, and enhancing project productivity and efficiency.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request, including the raw data collected during the experiment [wristband data (EDA), heart rate (HR), brain wave data (EEG), and virtual reality environment log files].
References
Akalin, N., A. Kristoffersson, and A. Loutfi. 2022. “Do you feel safe with your robot? Factors influencing perceived safety in human-robot interaction based on subjective and objective measures.” Int. J. Hum. Comput. Stud. 158 (Feb): 102744. https://doi.org/10.1016/j.ijhcs.2021.102744.
Akash, K., W. L. Hu, N. Jain, and T. Reid. 2018. “A classification model for sensing human trust in machines using EEG and GSR.” ACM Trans. Interact. Intell. Syst. 8 (4): 1–20. https://doi.org/10.1145/3132743.
Alakus, T. B., M. Gonen, and I. Turkoglu. 2020. “Database for an emotion recognition system based on EEG signals and various computer games–GAMEEMO.” Biomed. Signal Process. Control 60 (Jul): 101951. https://doi.org/10.1016/j.bspc.2020.101951.
Alarcon, G. M., A. M. Gibson, S. A. Jessup, and A. Capiola. 2021. “Exploring the differential effects of trust violations in human-human and human-robot interactions.” Appl. Ergon. 93 (May): 103350. https://doi.org/10.1016/j.apergo.2020.103350.
Appelhans, B. M., and L. J. Luecken. 2006. “Heart rate variability as an index of regulated emotional responding.” Rev. General Psychol. 10 (3): 229–240. https://doi.org/10.1037/1089-2680.10.3.229.
Arents, J., V. Abolins, J. Judvaitis, O. Vismanis, A. Oraby, and K. Ozols. 2021. “Human–robot collaboration trends and safety aspects: A systematic review.” J. Sens. Actuator Networks 10 (3): 48. https://doi.org/10.3390/jsan10030048.
Bach, D. R., G. Flandin, K. J. Friston, and R. J. Dolan. 2009. “Time-series analysis for rapid event-related skin conductance responses.” J. Neurosci. Methods 184 (2): 224–234. https://doi.org/10.1016/j.jneumeth.2009.08.005.
Baxter, P., P. Lightbody, and M. Hanheide. 2018. “Robots providing cognitive assistance in shared workspaces.” In Proc., Companion of the 2018 ACM/IEEE Int. Conf. on Human-Robot Interaction, 57–58. New York: Association for Computing Machinery.
Bieber, F., and J. Viehoff. 2022. “A paradigm-based explanation of trust.” Synthese 201 (1): 2. https://doi.org/10.1007/s11229-022-03993-4.
Butler, J. T., and A. Agah. 2001. “Psychological effects of behavior patterns of a mobile personal robot.” Auton. Robots 10 (2): 185–202. https://doi.org/10.1023/A:1008986004181.
Camara, F., and C. Fox. 2022. “Extending quantitative proxemics and trust to HRI.” In Proc., 2022 31st IEEE Int. Conf. on Robot and Human Interactive Communication (RO-MAN), 421–427. New York: IEEE.
Charalambous, G., S. Fletcher, and P. Webb. 2016. “The development of a scale to evaluate trust in industrial human-robot collaboration.” Int. J. Social Rob. 8 (2): 193–209. https://doi.org/10.1007/s12369-015-0333-8.
Chauhan, H., Y. Jang, S. Pradhan, and H. Moon. 2023. “Personalized optimal room temperature and illuminance for maximizing occupant’s mental task performance using physiological data.” J. Build. Eng. 78 (Nov): 107757. https://doi.org/10.1016/j.jobe.2023.107757.
Chen, M., S. Nikolaidis, H. Soh, D. Hsu, and S. Srinivasa. 2018. “Planning with trust for human-robot collaboration.” In Proc., 2018 ACM/IEEE Int. Conf. on Human-Robot Interaction, 307–315. New York: Association for Computing Machinery.
Choi, B., H. Jebelli, and S. Lee. 2019. “Feasibility analysis of electrodermal activity (EDA) acquired from wearable sensors to assess construction workers’ perceived risk.” Saf. Sci. 115 (Jun): 110–120. https://doi.org/10.1016/j.ssci.2019.01.022.
Cinaz, B., B. Arnrich, R. La Marca, and G. Tröster. 2013. “Monitoring of mental workload levels during an everyday life office-work scenario.” Pers. Ubiquitous Comput. 17 (2): 229–239. https://doi.org/10.1007/s00779-011-0466-1.
Cochran, C. D., and S. Urbanczyk. 1982. “The effect of availability of vertical space on personal space.” J. Psychol. 111 (1): 137–140. https://doi.org/10.1080/00223980.1982.9923525.
Dehais, F., E. A. Sisbot, R. Alami, and M. Causse. 2011. “Physiological and subjective evaluation of a human–robot object hand-over task.” Appl. Ergon. 42 (6): 785–791. https://doi.org/10.1016/j.apergo.2010.12.005.
Delgado, J. M. D., L. Oyedele, A. Ajayi, L. Akanbi, O. Akinade, M. Bilal, and H. Owolabi. 2019. “Robotics and automated systems in construction: Understanding industry-specific challenges for adoption.” J. Build. Eng. 26 (Nov): 100868. https://doi.org/10.1016/j.jobe.2019.100868.
De Visser, E. J., P. J. Beatty, J. R. Estepp, S. Kohn, A. Abubshait, J. R. Fedota, and C. G. McDonald. 2018. “Learning from the slips of others: Neural correlates of trust in automated agents.” Front. Hum. Neurosci. 12: 309. https://doi.org/10.3389/fnhum.2018.00309.
Golgouneh, A., and B. Tarvirdizadeh. 2020. “Fabrication of a portable device for stress monitoring using wearable sensors and soft computing algorithms.” Neural Comput. Appl. 32 (11): 7515–7537. https://doi.org/10.1007/s00521-019-04278-7.
Hack, N., et al. 2020. “Structural stay-in-place formwork for robotic in situ fabrication of non-standard concrete structures: A real scale architectural demonstrator.” Autom. Constr. 115 (Jul): 103197. https://doi.org/10.1016/j.autcon.2020.103197.
Hall, E. T. 1966. The hidden dimension. Garden City, NY: Doubleday.
Hancock, P. A., D. R. Billings, K. E. Schaefer, J. Y. Chen, E. J. De Visser, and R. Parasuraman. 2011. “A meta-analysis of factors affecting trust in human-robot interaction.” Hum. Factors 53 (5): 517–527. https://doi.org/10.1177/0018720811417254.
Hoff, K. A., and M. Bashir. 2015. “Trust in automation: Integrating empirical evidence on factors that influence trust.” Hum. Factors 57 (3): 407–434. https://doi.org/10.1177/0018720814547570.
Hopko, S., J. Wang, and R. Mehta. 2022. “Human factors considerations and metrics in shared space human-robot collaboration: A systematic review.” Front. Robot. AI 9: 799522. https://doi.org/10.3389/frobt.2022.799522.
Hostettler, D., S. Mayer, and C. Hildebrand. 2023. “Human-like movements of industrial robots positively impact observer perception.” Int. J. Social Robot. 15 (8): 1399–1417. https://doi.org/10.1007/s12369-022-00954-2.
ISO. 2016. Robots and robotic devices—Collaborative robots. ISO/TS 15066:2016(en). Geneva: ISO.
Jang, Y., K. Kim, F. Leite, S. Ayer, and Y. K. Cho. 2021. “Identifying the perception differences of emerging construction-related technologies between industry and academia to enable high levels of collaboration.” J. Constr. Eng. Manage. 147 (10): 06021004. https://doi.org/10.1061/(ASCE)CO.1943-7862.0002156.
Jeong, I., Y. Jang, J. Park, and Y. K. Cho. 2021. “Motion planning of mobile robots for autonomous navigation on uneven ground surfaces.” J. Comput. Civ. Eng. 35 (3): 04021001. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000963.
Karreman, D., L. Utama, M. Joosse, M. Lohse, B. van Dijk, and V. Evers. 2014. “Robot etiquette: How to approach a pair of people?” In Proc., 2014 ACM/IEEE Int. Conf. on Human-Robot Interaction, 196–197. New York: Association for Computing Machinery.
Khavas, Z. R., S. R. Ahmadzadeh, and P. Robinette. 2020. “Modeling trust in human-robot interaction: A survey.” In Proc., Social Robotics: 12th Int. Conf., ICSR 2020, 529–541. Golden, CO: Springer.
Kim, K., M. Kim, D. Kim, and D. Lee. 2019. “Modeling and velocity-field control of autonomous excavator with main control valve.” Automatica 104 (Jun): 67–81. https://doi.org/10.1016/j.automatica.2019.02.041.
Koay, K. L., E. A. Sisbot, D. S. Syrdal, M. L. Walters, K. Dautenhahn, and R. Alami. 2007. “Exploratory study of a robot approaching a person in the context of handing over an object.” In Proc., AAAI Spring Symp.: Multidisciplinary Collaboration for Socially Assistive Robotics, 18–24. Stanford, CA: Association for the Advancement of Artificial Intelligence.
Körber, M., E. Baseler, and K. Bengler. 2018. “Introduction matters: Manipulating trust in automation and reliance in automated driving.” Appl. Ergon. 66 (Jan): 18–31. https://doi.org/10.1016/j.apergo.2017.07.006.
Krausman, A., C. Neubauer, D. Forster, S. Lakhmani, A. L. Baker, S. M. Fitzhugh, G. Gremillion, J. L. Wright, J. S. Metcalfe, and K. E. Schaefer. 2022. “Trust measurement in human-autonomy teams: Development of a conceptual toolkit.” ACM Trans. Human-Robot Interact. 11 (3): 1–58. https://doi.org/10.1145/3530874.
Lagomarsino, M., M. Lorenzini, E. De Momi, and A. Ajoudani. 2022. “An online framework for cognitive load assessment in industrial tasks.” Rob. Comput. Integr. Manuf. 78 (Dec): 102380. https://doi.org/10.1016/j.rcim.2022.102380.
Lasota, P. A., and J. A. Shah. 2015. “Analyzing the effects of human-aware motion planning on close-proximity human–robot collaboration.” Hum. Factors 57 (1): 21–33. https://doi.org/10.1177/0018720814565188.
Lazanyi, K., and G. Maraczi. 2017. “Dispositional trust—Do we trust autonomous cars?” In Proc., 2017 IEEE 15th Int. Symp. on Intelligent Systems and Informatics (SISY), 135–140. New York: IEEE.
Le, A. 2021. The benefits and challenges of automation in the modular construction industry. San Luis Obispo, CA: California Polytechnic State Univ.
Lee, J., and N. Moray. 1992. “Trust, control strategies and allocation of function in human-machine systems.” Ergonomics 35 (10): 1243–1270. https://doi.org/10.1080/00140139208967392.
Lee, J. D., and N. Moray. 1994. “Trust, self-confidence, and operators’ adaptation to automation.” Int. J. Hum. Comput. Stud. 40 (1): 153–184. https://doi.org/10.1006/ijhc.1994.1007.
Lee, J. D., and K. A. See. 2004. “Trust in automation: Designing for appropriate reliance.” Hum. Factors 46 (1): 50–80. https://doi.org/10.1518/hfes.46.1.50.30392.
Leichtmann, B., and V. Nitsch. 2020. “How much distance do humans keep toward robots? Literature review, meta-analysis, and theoretical considerations on personal space in human-robot interaction.” J. Environ. Psychol. 68 (Apr): 101386. https://doi.org/10.1016/j.jenvp.2019.101386.
Leichtmann, B., V. Nitsch, and M. Mara. 2022. “Crisis ahead? Why human-robot interaction user studies may have replicability problems and directions for improvement.” Front. Robot. AI 9: 838116. https://doi.org/10.3389/frobt.2022.838116.
Liang, C. J., X. Wang, V. R. Kamat, and C. C. Menassa. 2021. “Human–robot collaboration in construction: Classification and research trends.” J. Constr. Eng. Manage. 147 (10): 03121006. https://doi.org/10.1061/(ASCE)CO.1943-7862.0002154.
Lischke, A., R. Jacksteit, A. Mau-Moeller, R. Pahnke, A. O. Hamm, and M. Weippert. 2018. “Heart rate variability is associated with psychosocial stress in distinct social domains.” J. Psychosomatic Res. 106: 56–61. https://doi.org/10.1016/j.jpsychores.2018.01.005.
Liu, Y., M. Habibnezhad, and H. Jebelli. 2021. “Brainwave-driven human-robot collaboration in construction.” Autom. Constr. 124 (Apr): 103556. https://doi.org/10.1016/j.autcon.2021.103556.
MacArthur, K. R., K. Stowers, and P. A. Hancock. 2017. “Human-robot interaction: Proximity and speed—slowly back away from the robot!” In Proc., AHFE 2016 Int. Conf. on Human Factors in Robots and Unmanned Systems: Advances in Human Factors in Robots and Unmanned Systems, 365–374. Berlin: Springer.
Malle, B. F., and D. Ullman. 2021. “A multidimensional conception and measure of human-robot trust.” Chap. 1 in Trust in human-robot interaction, 3–25. Cambridge, MA: Academic Press.
Matteucci, P., and F. Cepolina. 2015. “A robotic cutting tool for contaminated structure maintenance and decommissioning.” Autom. Constr. 58 (Oct): 109–117. https://doi.org/10.1016/j.autcon.2015.07.006.
Maurtua, I., A. Ibarguren, J. Kildal, L. Susperregi, and B. Sierra. 2017. “Human–robot collaboration in industrial applications: Safety, interaction and trust.” Int. J. Adv. Rob. Syst. 14 (4): 1729881417716010. https://doi.org/10.1177/1729881417716010.
Mayer, R. C., J. H. Davis, and F. D. Schoorman. 1995. “An integrative model of organizational trust.” Acad. Manage. Rev. 20 (3): 709–734. https://doi.org/10.2307/258792.
McCroskey, J. C. 1966. Scales for the measurement of ethos. Speech Monograph. London: Taylor & Francis.
Melenbrink, N., J. Werfel, and A. Menges. 2020. “On-site autonomous construction robots: Towards unsupervised building.” Autom. Constr. 119 (Nov): 103312. https://doi.org/10.1016/j.autcon.2020.103312.
Montague, E., J. Xu, and E. Chiou. 2014. “Shared experiences of technology and trust: An experimental study of physiological compliance between active and passive users in technology-mediated collaborative encounters.” IEEE Trans. Hum.-Mach. Syst. 44 (5): 614–624. https://doi.org/10.1109/THMS.2014.2325859.
Muir, B. M. 1987. “Trust between humans and machines, and the design of decision aids.” Int. J. Man Mach. Stud. 27 (5–6): 527–539. https://doi.org/10.1016/S0020-7373(87)80013-5.
Muir, B. M. 1994. “Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems.” Ergonomics 37 (11): 1905–1922. https://doi.org/10.1080/00140139408964957.
Mumm, J., and B. Mutlu. 2011. “Human-robot proxemics: Physical and psychological distancing in human-robot interaction.” In Proc., 6th Int. Conf. on Human-Robot Interaction, 331–338. New York: Association for Computing Machinery.
Nabulsi, S., and M. Armada. 2004. “Climbing strategies for remote maneuverability of ROBOCLIMBER.” In Proc., 35th Int. Symp. on Robotics, 23–26. Paris: International Federation of Robotics.
Natarajan, M., and M. Gombolay. 2020. “Effects of anthropomorphism and accountability on trust in human robot interaction.” In Proc., 2020 ACM/IEEE Int. Conf. on Human-Robot Interaction, 33–42. New York: Association for Computing Machinery.
Palmarini, R., I. F. del Amo, G. Bertolino, G. Dini, J. A. Erkoyuncu, R. Roy, and M. Farnsworth. 2018. “Designing an AR interface to improve trust in Human-Robots collaboration.” Procedia CIRP 70: 350–355. https://doi.org/10.1016/j.procir.2018.01.009.
Paluch, S., J. Wirtz, and W. H. Kunz. 2022. “The service robot revolution.” Chap. 20 in Research handbook on services management, 296. Cheltenham, UK: Edward Elgar Publishing.
Picard, R. W., S. Fedor, and Y. Ayzenberg. 2016. “Multiple arousal theory and daily-life electrodermal activity asymmetry.” Emotion Rev. 8 (1): 62–75. https://doi.org/10.1177/1754073914565517.
Plutchik, R. 1980. “A general psychoevolutionary theory of emotion.” In Theories of emotion, 3–33. New York: Academic Press.
Rios-Martinez, J., A. Spalanzani, and C. Laugier. 2015. “From proxemics theory to socially-aware navigation: A survey.” Int. J. Social Robot. 7 (2): 137–153. https://doi.org/10.1007/s12369-014-0251-1.
Robertson, R., C. Gockel, and E. Brauner. 2012. “Trust your teammates or bosses? Differential effects of trust on transactive memory, job satisfaction, and performance.” Employee Relations 35 (2): 222–242. https://doi.org/10.1108/01425451311287880.
Schaefer, K. E. 2016. “Measuring trust in human robot interactions: Development of the “trust perception scale-HRI.” In Robust intelligence and trust in autonomous systems, 191–218. Boston: Springer.
Sebo, S. S., P. Krishnamurthi, and B. Scassellati. 2019. “‘I don’t believe you’: Investigating the effects of robot trust violation and repair.” In Proc., 2019 14th ACM/IEEE Int. Conf. on Human-Robot Interaction (HRI), 57–65. New York: IEEE.
Stollnberger, G., A. Weiss, and M. Tscheligi. 2013. “‘The harder it gets’ Exploring the interdependency of input modalities and task complexity in human-robot collaboration.” In Proc., 2013 IEEE RO-MAN, 264–269. New York: IEEE.
Stork, S., C. Stossel, and A. Schubo. 2008. “Optimizing human-machine interaction in manual assembly.” In Proc., RO-MAN 2008—The 17th IEEE Int. Symp. on Robot and Human Interactive Communication, 113–118. New York: IEEE.
Story, M., P. Webb, S. R. Fletcher, G. Tang, C. Jaksic, and J. Carberry. 2022. “Do speed and proximity affect human-robot collaboration with an industrial robot arm?” Int. J. Social Robot. 14 (4): 1087–1102. https://doi.org/10.1007/s12369-021-00853-y.
Sundstrom, E. 1975. “An experimental study of crowding: Effects of room size, intrusion, and goal blocking on nonverbal behavior, self-disclosure, and self-reported stress.” J. Personality Social Psychol. 32 (4): 645. https://doi.org/10.1037/0022-3514.32.4.645.
Takayama, L., and C. Pantofaru. 2009. “Influences on proxemic behaviors in human-robot interaction.” In Proc., 2009 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, 5495–5502. New York: IEEE.
Toichoa Eyam, A., W. M. Mohammed, and J. L. Martinez Lastra. 2021. “Emotion-driven analysis and control of human-robot interactions in collaborative applications.” Sensors 21 (14): 4626. https://doi.org/10.3390/s21144626.
Torta, E., J. Oberzaucher, F. Werner, R. H. Cuijpers, and J. F. Juola. 2013. “Attitudes towards socially assistive robots in intelligent homes: Results from laboratory studies and field trials.” J. Human-Robot Interact. 1 (2): 76–99. https://doi.org/10.5898/JHRI.1.2.Torta.
Wagner-Hartl, V., R. Schmid, and K. Gleichauf. 2022. “The influence of task complexity on acceptance and trust in human-robot interaction–gender and age differences.” Cognitive Comput. Internet Things 43: 118–126. https://doi.org/10.54941/ahfe1001846.
Walters, M. L., M. A. Oskoei, D. S. Syrdal, and K. Dautenhahn. 2011. “A long-term human-robot proxemic study.” In Proc., 2011 RO-MAN, 137–142. New York: IEEE.
Wang, W., Y. Chen, R. Li, and Y. Jia. 2019. “Learning and comfort in human–robot interaction: A review.” Appl. Sci. 9 (23): 5152. https://doi.org/10.3390/app9235152.
Winfield, A. 2012. Robotics: A very short introduction. Oxford, UK: Oxford University Press.
Wojton, H. M., D. Porter, S. T. Lane, C. Bieber, and P. Madhavan. 2020. “Initial validation of the trust of automated systems test (TOAST).” J. Social Psychol. 160 (6): 735–750. https://doi.org/10.1080/00224545.2020.1749020.
Wu, B., B. Hu, and H. Lin. 2017. “Toward efficient manufacturing systems: A trust based human robot collaboration.” In Proc., 2017 American Control Conf. (ACC), 1536–1541. New York: IEEE.
Yan, Z., and S. Holtmanns. 2008. “Trust modeling and management: From social trust to digital trust.” In Computer security, privacy and politics: Current issues, challenges and solutions, 290–323. Hershey, PA: IGI Global.
Young, J. E., Y. Kamiyama, J. Reichenbach, T. Igarashi, and E. Sharlin. 2011. “How to walk a robot: A dog-leash human-robot interface.” In Proc., 2011 RO-MAN, 376–382. New York: IEEE.
Zhang, B., Y. Morère, L. Sieler, C. Langlet, B. Bolmont, and G. Bourhis. 2017. “Reaction time and physiological signals for stress recognition.” Biomed. Signal Process. Control 38 (Sep): 100–107. https://doi.org/10.1016/j.bspc.2017.05.003.
Zhang, Q., T. Liu, Z. Zhang, Z. Huangfu, Q. Li, and Z. An. 2019. “Unmanned rolling compaction system for rockfill materials.” Autom. Constr. 100 (Apr): 103–117. https://doi.org/10.1016/j.autcon.2019.01.004.
Information & Authors
Information
Published In
Journal of Computing in Civil Engineering
Volume 38 • Issue 4 • July 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Sep 3, 2023
Accepted: Jan 9, 2024
Published online: Apr 15, 2024
Published in print: Jul 1, 2024
Discussion open until: Sep 15, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.