Exploring Human-Machine Interfaces for Teleoperation of Excavator
Publication: Construction Research Congress 2022
ABSTRACT
With the advancement of automation and robotic technologies, the teleoperation has been leveraged as a promising solution for human workers in a hazardous construction work environment. Since human operators and construction sites are separated in a distance, teleoperation requires a seamless human-machine interface, an intermediate medium, to communicate between humans and machines in construction sites. Several types of teleoperation interfaces including conventional joysticks, haptic devices, graphic user interfaces, auditory interfaces, and tactile interfaces have been developed to control and command construction robotics remotely. The ultimate goal of human-machine interfaces for remote operations is to make intuitive sensory channels that can provide and receive enough information while reducing the associated cognitive and physical load on human operators. Previously developed interfaces have tried to achieve such goals, but each interface still has challenges that should be assessed for enhancing the future teleoperation application in construction workplaces. This paper examines different human-machine interfaces for excavator teleoperation in terms of its on-site usability and intuitiveness. The capabilities of the interfaces for excavator teleoperation are evaluated based on their limitations and requirements. The outcome is expected to provide better understanding of teleoperation interfaces for excavators and guiding future directions for addressing underlying challenges.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Bhalerao, B. N., Dunston, P. S., and Proctor, R. W. (2017). “Use of PC-based Simulators to Train Basic Control Functions of a Hydraulic Excavator: Audiovisual Instruction Contrasted with Hands-On Exploration.” International Journal of Human–Computer Interaction, Taylor & Francis, 33(1), 66–74.
Chao, G. (2009). “Human-Computer Interaction: Process and Principles of Human-Computer Interface Design.” 2009 International Conference on Computer and Automation Engineering, 230–233.
Dadhich, S., Bodin, U., and Andersson, U. (2016). “Key challenges in automation of earth-moving machines.” Automation in Construction, 68, 212–222.
Gong, D., Wang, Y., Yu, J., and Zuo, G. (2019). “Motion Mapping from a Human Arm to a Heterogeneous Excavator-like Robotic Arm for Intuitive Teleoperation.” 2019 IEEE International Conference on Real-time Computing and Robotics (RCAR), 493–498.
Hashimoto, T., Yamauchi, G., Fujino, K., Yuta, S., and Tateyama, K. (2018). “Study of Operator’s Line of Sight in Unmanned Construction Systems.” 2018 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), 1–6.
Hemami, A., and Hassani, F. (2009). “An overview of autonomous loading of bulk material.” 26th International Symposium on Automation and Robotics in Construction, Citeseer, 405–411.
Hong, Z., Zhang, Q., Su, X., and Zhang, H. (2020). “Effect of virtual annotation on performance of construction equipment teleoperation under adverse visual conditions.” Automation in Construction, 118, 103296.
Ito, M., Raima, C., Saiki, S., Yamazaki, Y., and Kurita, Y. (2021). “Effects of Machine Instability Feedback on Safety During Digging Operation in Teleoperated Excavators.” IEEE Access, 9, 28987–28998.
Kim, D., Kim, J., Lee, K., Park, C., Song, J., and Kang, D. (2009). “Excavator tele-operation system using a human arm.” Automation in Construction, 18(2), 173–182.
Mavridis, N., Pierris, G., Gallina, P., Papamitsiou, Z., and Saad, U. (2015). “On the subjective difficulty of Joystick-based robot arm teleoperation with auditory feedback.” 2015 IEEE 8th GCC Conference Exhibition, 1–6.
Moher, D., Liberati, A., Tetzlaff, J., and Altman, D. G. (2010). “Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement.” International Journal of Surgery, 8(5), 336–341.
Nagano, H., Takenouchi, H., Cao, N., Konyo, M., and Tadokoro, S. (2020). “Tactile feedback system of high-frequency vibration signals for supporting delicate teleoperation of construction robots.” Advanced Robotics, Taylor & Francis, 34(11), 730–743.
Okishiba, S., Fukui, R., Takagi, M., Azumi, H., Warisawa, S., Togashi, R., Kitaoka, H., and Ooi, T. (2019). “Tablet interface for direct vision teleoperation of an excavator for urban construction work.” Automation in Construction, 102, 17–26.
Sato, R., Kamezaki, M., Niuchi, S., Sugano, S., and Iwata, H. (2020). “Cognitive untunneling multi-view system for teleoperators of heavy machines based on visual momentum and saliency.” Automation in Construction, 110, 103047.
Scalera, L., Seriani, S., Gallina, P., Luca, M. D., and Gasparetto, A. (2018). “An Experimental Setup to Test Dual-Joystick Directional Responses to Vibrotactile Stimuli.” IEEE Transactions on Haptics, 11(3), 378–387.
Shin, D., Kang, M., Lee, S., and Han, C. (2012). “Development of remote controlled manipulation device for a conventional excavator without renovation.” 2012 IEEE/SICE International Symposium on System Integration (SII), 546–551.
Wallmyr, M., Sitompul, T. A., Holstein, T., and Lindell, R. (2019). “Evaluating Mixed Reality Notifications to Support Excavator Operator Awareness.” Human-Computer Interaction – INTERACT 2019, Lecture Notes in Computer Science, D. Lamas, F. Loizides, L. Nacke, H. Petrie, M. Winckler, and P. Zaphiris, eds., Springer International Publishing, Cham, 743–762.
Zhang, Y., Liu, H., Kang, S.-C., and Al-Hussein, M. (2020). “Virtual reality applications for the built environment: Research trends and opportunities.” Automation in Construction, 118, 103311.
Zhou, T., Zhu, Q., and Du, J. (2020). “Intuitive robot teleoperation for civil engineering operations with virtual reality and deep learning scene reconstruction.” Advanced Engineering Informatics, 46, 101170.
Information & Authors
Information
Published In
Construction Research Congress 2022
Pages: 757 - 765
History
Published online: Mar 7, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.