Benchmarking Human versus Robot Performance in Emergency Structural Inspection
Publication: Journal of Construction Engineering and Management
Volume 148, Issue 8
Abstract
Human–robot collaboration (HRC) has gained significant momentum in various civil engineering applications. A representative application is robot-assisted emergency structural inspection, i.e., a human–robot partnered on-site investigation of structural damage and possible causes after any human-caused or natural incidents. Human experts are good at making informed decisions that reflect structural failure patterns, whereas robots can help in searching larger areas and in hazardous places. An effective HRC team would leverage the advantages of human and robot agents. However, deeper insights and direct evidence are still needed to build a baseline model, and in particular to identify the relative advantages and limitations of human and robot agents in emergency structural inspections. To establish a basis for efficient HRC emergency inspection team design, this paper presents a benchmark to compare human and robot performance in a simulated emergency structural inspection task following an earthquake disaster. The total amount of identified structural damage, inspection time, and route patterns of humans and robots are compared. The results show that humans outperformed autonomous robots in most inspection metrics, possibly owing to prior knowledge about the patterns of structural failure, and presented a different but more efficient route pattern. The findings are expected to inspire better HRC team design for emergency structural inspection tasks and other relevant knowledge-based HRC tasks.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
The data used in this research can be found at: https://drive.google.com/drive/folders/1r7y6b9NTobh0whyF3OIGn6ETAM-lJxY9?usp=sharing.
Acknowledgments
This material was partially supported by the National Science Foundation (NSF) under Grants 2033592 and 1937878, as well as the National Institute of Standards and Technology (NIST) under Grants 60NANB18D152C and 70NANB21H045. The findings do not reflect the views of NSF or NIST.
References
Abbass, H. A. 2019. “Social integration of artificial intelligence: Functions, automation allocation logic and human-autonomy trust.” Cognit. Comput. 11 (2): 159–171. https://doi.org/10.1007/s12559-018-9619-0.
Agnisarman, S., S. Lopes, K. C. Madathil, K. Piratla, and A. Gramopadhye. 2019. “A survey of automation-enabled human-in-the-loop systems for infrastructure visual inspection.” Autom. Constr. 97 (Jan): 52–76. https://doi.org/10.1016/j.autcon.2018.10.019.
Ajoudani, A., A. M. Zanchettin, S. Ivaldi, A. Albu-Schäffer, K. Kosuge, and O. Khatib. 2018. “Progress and prospects of the human–robot collaboration.” Autom. Rob. 42 (5): 957–975. https://doi.org/10.1007/s10514-017-9677-2.
Arslan, M., H. H. Korkmaz, and F. Gulay. 2006. “Damage and failure pattern of prefabricated structures after major earthquakes in Turkey and shortfalls of the Turkish Earthquake code.” Eng. Fail. Anal. 13 (4): 537–557. https://doi.org/10.1016/j.engfailanal.2005.02.006.
Asadi, K., A. K. Suresh, A. Ender, S. Gotad, S. Maniyar, S. Anand, M. Noghabaei, K. Han, E. Lobaton, and T. Wu. 2020. “An integrated UGV-UAV system for construction site data collection.” Autom. Constr. 112 (Apr): 103068. https://doi.org/10.1016/j.autcon.2019.103068.
Awad, R., M. Fechter, and J. van Heerden. 2017. “Integrated risk assessment and safety consideration during design of HRC workplaces.” In Proc., 22nd IEEE Int. Conf. on Emerging Technologies and Factory Automation (ETFA), 1–10. New York: IEEE.
Bailey, T., and H. Durrant-Whyte. 2006. “Simultaneous localization and mapping (SLAM): Part II.” IEEE Rob. Autom. Mag. 13 (3): 108–117. https://doi.org/10.1109/MRA.2006.1678144.
Bloch, T., R. Sacks, and O. Rabinovitch. 2016. “Interior models of earthquake damaged buildings for search and rescue.” Adv. Eng. Inf. 30 (1): 65–76. https://doi.org/10.1016/j.aei.2015.12.001.
Chakraborti, T., S. Kambhampati, M. Scheutz, and Y. Zhang. 2017. “AI challenges in human-robot cognitive teaming.” Preprint, submitted July 15, 2017. http://arxiv.org/abs/1707.04775.
Chan, D. W., H. T. Hung, A. P. Chan, and T. K. Lo. 2014. “Overview of the development and implementation of the mandatory building inspection scheme (MBIS) in Hong Kong.” Built Environ. Project Asset Manage. 4 (1): 71–89. https://doi.org/10.1108/BEPAM-07-2012-0040.
Chen, S., D. F. Laefer, E. Mangina, S. I. Zolanvari, and J. Byrne. 2019. “UAV bridge inspection through evaluated 3D reconstructions.” J. Bridge Eng. 24 (4): 05019001. https://doi.org/10.1061/(ASCE)BE.1943-5592.0001343.
Chien, S.-Y., H. Wang, and M. Lewis. 2010. “Human vs. algorithmic path planning for search and rescue by robot teams.” In Proc., Human Factors and Ergonomics Society Annual Meeting, 379–383. New York: SAGE.
Cummings, M. L., J. J. Marquez, and N. Roy. 2012. “Human-automated path planning optimization and decision support.” Int. J. Hum. Comput. Stud. 70 (2): 116–128. https://doi.org/10.1016/j.ijhcs.2011.10.001.
Dang, J., D. Haruta, A. Shrestha, H. Endo, S. Matsunaga, A. Kasai, and X. Wang. 2017. “Arial patrol for bridge routine and post earthquake emergency inspection using small aerial photography UAV.” In Proc., 5th Int. Symp. on Advances in Civil and Environmental Engineering Practises for Sustainable Development, 21. Galle, Sri Lanka: Univ. of Ruhuna.
Du, J., Y. Shi, C. Mei, J. Quarles, and W. Yan. 2016. “Communication by interaction: A multiplayer VR environment for building walkthroughs.” In Proc., Construction Research Congress 2016, 2281–2290. Reston, VA: ASCE.
Du, J., Y. Shi, Z. Zou, and D. Zhao. 2018a. “CoVR: Cloud-based multiuser virtual reality headset system for project communication of remote users.” J. Constr. Eng. Manage. 144 (2): 04017109. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001426.
Du, J., Z. Zou, Y. Shi, and D. Zhao. 2017. “Simultaneous data exchange between BIM and VR for collaborative decision making.” In Computing in civil engineering 2017, 1–8. Reston, VA: ASCE.
Du, J., Z. Zou, Y. Shi, and D. Zhao. 2018b. “Zero latency: Real-time synchronization of BIM data in virtual reality for collaborative decision-making.” Autom. Constr. 85 (Jan): 51–64. https://doi.org/10.1016/j.autcon.2017.10.009.
ElectricNightOwl. 2017. “ArmSwinger.” Accessed August 30, 2021. https://github.com/ElectricNightOwl/ArmSwinger.
Fong, T., and I. Nourbakhsh. 2005. “Interaction challenges in human-robot space exploration.” Interactions 12 (2): 42–45. https://doi.org/10.1145/1052438.1052462.
Georgescu, E.-S., and A. Pomonis. 2012. “Building damage vs. territorial casualty patterns during the Vrancea (Romania) earthquakes of 1940 and 1977.” In Proc., 15th World Conf. on Earthquake Engineering, 24–28. Lisboa, Portugal: Sociedade Portuguesa de Engenharia Sismica.
Girden, E. R. 1992. ANOVA: Repeated measures. New York: SAGE.
Gong, W., X. Xie, and Y.-J. Liu. 2018. “Human experience–inspired path planning for robots.” Int. J. Adv. Rob. Syst. 15 (1): 1729881418757046.
Grisetti, G., R. Kümmerle, C. Stachniss, and W. Burgard. 2010. “A tutorial on graph-based SLAM.” IEEE Intell. Transp. Syst. Mag. 2 (4): 31–43. https://doi.org/10.1109/MITS.2010.939925.
Guo, Z., and B. Li. 2017. “Evolutionary approach for spatial architecture layout design enhanced by an agent-based topology finding system.” Front. Archit. Res. 6 (1): 53–62. https://doi.org/10.1016/j.foar.2016.11.003.
Gusev, A., M. Radulian, M. Rizescu, and G. Panza. 2002. “Source scaling of intermediate-depth Vrancea earthquakes.” Geophys. J. Int. 151 (3): 879–889. https://doi.org/10.1046/j.1365-246X.2002.01816.x.
Guzzetti, D., and H. Baoyin. 2019. “Human path-planning for autonomous spacecraft guidance at binary asteroids.” IEEE Trans. Aerosp. Electron. Syst. 55 (6): 3126–3138. https://doi.org/10.1109/TAES.2019.2899795.
Hasauino. 2017. “rrt exploration.” Accessed November 21, 2021. https://github.com/hasauino/rrt_exploration.
Heyer, C. 2010. “Human-robot interaction and future industrial robotics applications.” In Proc., 2010 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, 4749–4754. New York: IEEE.
Ibrahim, A., A. Sabet, and M. Golparvar-Fard. 2019. “BIM-driven mission planning and navigation for automatic indoor construction progress detection using robotic ground platform.” In Proc., European Conf. on Computing in Construction, 10–12. Dublin, Ireland: Univ. College Dublin.
Kim, P., L. C. Price, J. Park, and Y. K. Cho. 2019. “UAV-UGV cooperative 3D environmental mapping.” In Proc., Computing in Civil Engineering 2019: Data, Sensing, and Analytics, 384–392. Reston, VA: ASCE.
Lee, K.-Y., S.-Y. Lee, J.-H. Choi, S.-H. Lee, and C.-S. Han. 2006. “The application of the human-robot cooperative system for construction robot manipulating and installing heavy materials.” In Proc., 2006 SICE-ICASE Int. Joint Conf., 4798–4802. New York: IEEE.
Lin, T.-H., J.-T. Huang, and A. Putranto. 2021. “Integrated smart robot with earthquake early warning system for automated inspection and emergency response.” Nat. Hazard. 10 (1): 765–786.
Mader, D., R. Blaskow, P. Westfeld, and C. Weller. 2016. “Potential of UAV-based laser scanner and multispectral camera data in building inspection.” Int. Arch. Photogramm. Rem. Sens. Spatial Inf. Sci. 41 (Jul): 1135–1142.
Marquez, J., M. Feary, J. Zumbado, and D. Billman. 2013. Risk of inadequate design of human and automation/robotic integration: Evidence report. Washington, DC: National Aeronautics and Space Administration.
McKnight, P. E., and J. Najab. 2010. “Mann-Whitney U test.” In The Corsini encyclopedia of psychology. New York: Wiley.
Meriam, J. L., L. G. Kraige, and J. N. Bolton. 2020. Engineering mechanics: Dynamics. New York: Wiley.
Miura, H., A. Watanabe, M. Okugawa, T. Miura, and T. Koganeya. 2020. “Plant inspection by using a ground vehicle and an aerial robot: Lessons learned from plant disaster prevention challenge in world robot summit 2018.” Adv. Rob. 34 (2): 104–118. https://doi.org/10.1080/01691864.2019.1690575.
Mohanta, J. C., and A. Keshari. 2019. “A knowledge based fuzzy-probabilistic roadmap method for mobile robot navigation.” Appl. Soft Comput. 79 (Jun): 391–409. https://doi.org/10.1016/j.asoc.2019.03.055.
Qi, C. R., H. Su, K. Mo, and L. J. Guibas. 2017. “Pointnet: Deep learning on point sets for 3d classification and segmentation.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 652–660. New York: IEEE.
Qin, H., Z. Meng, W. Meng, X. Chen, H. Sun, F. Lin, and M. H. Ang. 2019. “Autonomous exploration and mapping system using heterogeneous UAVs and UGVs in GPS-denied environments.” IEEE Trans. Veh. Technol. 68 (2): 1339–1350. https://doi.org/10.1109/TVT.2018.2890416.
Rahman, S. M., Z. Liao, L. Jiang, and Y. Wang. 2016. “A regret-based autonomy allocation. scheme for human-robot shared vision systems in collaborative assembly in manufacturing.” In Proc., IEEE Int. Conf. on Automation Science and Engineering (CASE), 897–902. New York: IEEE.
Rakha, T., and A. Gorodetsky. 2018. “Review of Unmanned Aerial System (UAS) applications in the built environment: Towards automated building inspection procedures using drones.” Autom. Constr. 93 (Sep): 252–264. https://doi.org/10.1016/j.autcon.2018.05.002.
Richter, K.-F. 2009. “Adaptable path planning in regionalized environments.” In Proc., Int. Conf. on Spatial Information Theory, 453–470. New York: Springer.
Roncone, A., O. Mangin, and B. Scassellati. 2017. “Transparent role assignment and task allocation in human robot collaboration.” In Proc., 2017 IEEE Int. Conf. on Robotics and Automation (ICRA), 1014–1021. New York: IEEE.
ROSwiki. 2018. “rrt_exploration.” Accessed June 10, 2021. http://wiki.ros.org/rrt_exploration.
ROSwiki. 2019. “gmapping.” Accessed June 10, 2021. http://wiki.ros.org/gmapping.
Schmid, K. F. 2014. Building inspection manual: A guide for building professionals for maintenance, safety, and assessment. New York: Momentum Press.
Shapiro, S. S., and M. B. Wilk. 1965. “An analysis of variance test for normality (complete samples).” Biometrika 52 (3/4): 591–611. https://doi.org/10.2307/2333709.
Sharma, K., L. Deng, and C. C. Noguez. 2016. “Field investigation on the performance of building structures during the April 25, 2015, Gorkha earthquake in Nepal.” Eng. Struct. 121 (Aug): 61–74. https://doi.org/10.1016/j.engstruct.2016.04.043.
Shi, Y., J. Du, and E. Ragan. 2020. “Review visual attention and spatial memory in building inspection: Toward a cognition-driven information system.” Adv. Eng. Inf. 44 (Apr): 101061. https://doi.org/10.1016/j.aei.2020.101061.
Shi, Y., J. Du, E. Ragan, K. Choi, and S. Ma. 2018. “Social influence on construction safety behaviors: A multi-user virtual reality experiment.” In Proc., Construction Research Congress, 147–183. Reston, VA: ASCE.
SIMULIA. 2021. “ABAQUS UNIFIED FEA.” Accessed June 10, 2021. https://www.3ds.com/products-services/simulia/products/abaqus/.
Skeete, J.-P. 2018. “Level 5 autonomy: The new face of disruption in road transport.” Technol. Forecast. Soc. Change 134 (Sep): 22–34. https://doi.org/10.1016/j.techfore.2018.05.003.
Theodorou, A., R. H. Wortham, and J. J. Bryson. 2016. “Why is my robot behaving like that? Designing transparency for real time inspection of autonomous robots.” In Proc., AISB Workshop on Principles of Robotics. Nagykovácsi, Hungary: American International School of Budapest.
Tobii. 2021. “Pioneers of eye tracking and attention computing.” Accessed June 10, 2021. https://www.tobii.com/group/about/.
Turner, D., and D. Scardino. 2018. “Emergency structural engineering support of a fire origin and cause analysis.” In Proc., Forensic Engineering 2018: Forging Forensic Frontiers, 386–395. Reston, VA: ASCE.
Umari, H. A.-R. 2017. “Multi-robot map exploration based on multiple rapidly-exploring randomized trees.” Doctoral dissertation, Dept. of Mechanical Engineering, American Univ. of Sharjah.
Vonásek, V., J. Faigl, T. Krajník, and L. Přeučil. 2009. “RRT-path–a guided rapidly exploring random tree.” In Robot motion and control 2009, 307–316. New York: Springer.
Wiener, J. M., M. Lafon, and A. Berthoz. 2008. “Path planning under spatial uncertainty.” Mem. Cognit. 36 (3): 495–504. https://doi.org/10.3758/MC.36.3.495.
Wikipedia. 2021. “Analysis of variance.” Accessed November 10, 2021. https://en.wikipedia.org/wiki/Analysis_of_variance.
Zhao, B., F. Taucer, and T. Rossetto. 2009. “Field investigation on the performance of building structures during the 12 May 2008 Wenchuan earthquake in China.” Eng. Struct. 31 (8): 1707–1723. https://doi.org/10.1016/j.engstruct.2009.02.039.
Zhou, T., Q. Zhu, and J. Du. 2020. “Intuitive robot teleoperation for civil engineering operations with virtual reality and deep learning scene reconstruction.” Adv. Eng. Inf. 46 (Oct): 101170. https://doi.org/10.1016/j.aei.2020.101170.
Zomeren, M. V., J. M. Peschel, T. Mann, G. Knezek, J. Doebbler, J. Davis, T. A. Hammond, A. H. Oomes, and R. R. Murphy. 2009. “Human-robot interaction observations from a proto-study using SUAVs for structural inspection.” In Proc., 4th ACM/IEEE Int. Conf. on Human Robot Interaction, 235–236. New York: IEEE.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 148 • Issue 8 • August 2022
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Sep 13, 2021
Accepted: Mar 30, 2022
Published online: Jun 2, 2022
Published in print: Aug 1, 2022
Discussion open until: Nov 2, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Pengxiang Xia, Tianyu Zhou, Yang Ye, Jing Du, Human Autonomy Teaming for ROV Shared Control, Journal of Computing in Civil Engineering, 10.1061/JCCEE5.CPENG-5756, 38, 4, (2024).
- Yang Ye, Tianyu Zhou, Jing Du, Robot-Assisted Immersive Kinematic Experience Transfer for Welding Training, Journal of Computing in Civil Engineering, 10.1061/JCCEE5.CPENG-5138, 37, 2, (2023).