Haptic Augmentation System for Construction Robot Teleoperation
Publication: Computing in Civil Engineering 2023
ABSTRACT
Robot teleoperation has gained popularity in a variety of construction applications. A significant challenge of robot teleoperation is the disconnection between the robotic perception and the human operator’s sensory processes, creating a sensorimotor mismatch in motor-intensive activities. As a result, recent robot teleoperation literature has started investigating haptic interactions in the feedback and control loop. Haptics, including the feeling of force, motion, and vibrations, have been proven helpful for accurately perceiving task status and physical systems with heavy motions and enabling precision operations. However, the current haptic simulation methods are oversimplified and hard to provide a comprehensive, realistic, and complete simulation of the physical interaction that occurs in typical construction operations. This study aims to test a comprehensive, high-fidelity physical rendering method that simulates complete real-world physical processes via the physics engine. The proposed method captures all categories of physical interaction in typical motor-intensive construction tasks, including weight force, texture force, inertia force, and impact force. The result shows that the proposed method builds a digital twin model of real physical processes to facilitate intuitive robot teleoperation.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Bekiroglu, Y., Laaksonen, J., Jorgensen, J. A., Kyrki, V., and Kragic, D. (2011). “Assessing grasp stability based on learning and haptic data.” IEEE Transactions on Robotics, 27(3), 616–629.
Bimbo, J., Pacchierotti, C., Aggravi, M., Tsagarakis, N., and Prattichizzo, D. “Teleoperation in cluttered environments using wearable haptic feedback.” Proc., 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, 3401–3408.
Boessenkool, H., Abbink, D., Heemskerk, C., Steinbuch, M., De Baar, M., Wildenbeest, J., Ronden, D., and Koning, J. (2013). “Analysis of human-in-the-loop tele-operated maintenance inspection tasks using VR.” Fusion Engineering and Design, 88(9-10), 2164–2167.
Charness, G., Gneezy, U., and Kuhn, M. A. (2012). “Experimental methods: Between-subject and within-subject design.” Journal of Economic Behavior & Organization, 81(1), 1–8.
Du, J., Shi, Y., Zou, Z., and Zhao, D. (2018). “CoVR: Cloud-based multiuser virtual reality headset system for project communication of remote users.” Journal of Construction Engineering and Management, 144(2), 04017109.
Du, J., Zou, Z., Shi, Y., and Zhao, D. (2018). “Zero latency: Real-time synchronization of BIM data in virtual reality for collaborative decision-making.” Automation in Construction, 85, 51–64.
Edirisinghe, R. (2019). “Digital skin of the construction site.” Engineering, Construction and Architectural Management, 26(2), 184–223.
Endsley, M. R. (2017). “Toward a theory of situation awareness in dynamic systems.” Situational awareness, Routledge, 9–42.
Görsch, C., Seppänen, O., Peltokorpi, A., and Lavikka, R. “Construction workers’situational awareness–an overlooked perspective.” Proc., Annual Conference of the International Group for Lean Construction, IGLC. net, 937–948.
Hart, S. G., and Staveland, L. E. (1988). “Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research.” Advances in psychology, Elsevier, 139–183.
Hokayem, P. F., and Spong, M. W. (2006). “Bilateral teleoperation: An historical survey.” Automatica, 42(12), 2035–2057.
Horie, A., Saraiji, M. Y., Kashino, Z., and Inami, M. “EncounteredLimbs: A Room-scale Encountered-type Haptic Presentation using Wearable Robotic Arms.” Proc., 2021 IEEE Virtual Reality and 3D User Interfaces (VR), IEEE, 260–269.
Lathan, C. E., and Tracey, M. (2002). “The effects of operator spatial perception and sensory feedback on human-robot teleoperation performance.” Presence: Teleoperators & virtual environments, 11(4), 368–377.
Lederman, S. J., and Klatzky, R. L. (2009). “Haptic perception: A tutorial.” Attention, Perception, & Psychophysics, 71(7), 1439–1459.
Li, S., Rameshwar, R., Votta, A. M., and Onal, C. D. (2019). “Intuitive control of a robotic arm and hand system with pneumatic haptic feedback.” Ieee Robotics and Automation Letters, 4(4), 4424–4430.
Martin, S., and Hillier, N. “Characterisation of the Novint Falcon haptic device for application as a robot manipulator.” Proc., Australasian Conference on Robotics and Automation (ACRA), Citeseer, 291–292.
Merritt, S. M. (2011). “Affective Processes in Human–Automation Interactions.” Human Factors: The Journal of the Human Factors and Ergonomics Society, 53(4), 356–370.
Niemeyer, G., Preusche, C., Stramigioli, S., and Lee, D. (2016). “Telerobotics.” Springer handbook of robotics, Springer, 1085–1108.
Pacchierotti, C., Ongaro, F., van den Brink, F., Yoon, C., Prattichizzo, D., Gracias, D. H., and Misra, S. (2017). “Steering and control of miniaturized untethered soft magnetic grippers with haptic assistance.” IEEE transactions on automation science and engineering, 15(1), 290–306.
Paquet, V., Punnett, L., Woskie, S., and Buchholz, B. (2005). “Reliable exposure assessment strategies for physical ergonomics stressors in construction and other non-routinized work.” Ergonomics, 48(9), 1200–1219.
Pinto, L., and Gupta, A. “Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours.” Proc., 2016 IEEE international conference on robotics and automation (ICRA), IEEE, 3406–3413.
Pittman, C., and LaViola, J. J., Jr. “Exploring head tracked head mounted displays for first person robot teleoperation.” Proc., Proceedings of the 19th international conference on Intelligent User Interfaces, 323–328.
Roja, Z., Kalkis, H., Reinholds, I., and Cekuls, A. (2016). “Ergonomics risk analysis in construction operations.” Agronomy Research, 14(1), 211–219.
Shimojo, M., Araki, T., Ming, A., and Ishikawa, M. (2010). “A high-speed mesh of tactile sensors fitting arbitrary surfaces.” IEEE Sensors Journal, 10(4), 822–830.
Singh, J., Srinivasan, A. R., Neumann, G., and Kucukyilmaz, A. (2020). “Haptic-guided teleoperation of a 7-dof collaborative robot arm with an identical twin master.” IEEE transactions on haptics, 13(1), 246–252.
Taylor, R. M. (2017). “Situational awareness rating technique (SART): The development of a tool for aircrew systems design.” Situational awareness, Routledge, 111–128.
Villani, V., Pini, F., Leali, F., and Secchi, C. (2018). “Survey on human–robot collaboration in industrial settings: Safety, intuitive interfaces and applications.” Mechatronics, 55, 248–266.
Zhang, Y., Chen, B. K., Liu, X., and Sun, Y. (2009). “Autonomous robotic pick-and-place of microobjects.” IEEE Transactions on Robotics, 26(1), 200–207.
Zhou, T., Zhu, Q., and Du, J. (2020). “Intuitive robot teleoperation for civil engineering operations with virtual reality and deep learning scene reconstruction.” Advanced Engineering Informatics, 46, 101170.
Zhu, Q., Zhou, T., and Du, J. (2022). “Upper-body haptic system for snake robot teleoperation in pipelines.” Advanced Engineering Informatics, 51, 101532.
Information & Authors
Information
Published In
Computing in Civil Engineering 2023
Pages: 496 - 504
History
Published online: Jan 25, 2024
ASCE Technical Topics:
- Automation and robotics
- Bibliographies
- Business management
- Construction engineering
- Construction management
- Construction methods
- Continuum mechanics
- Dynamics (solid mechanics)
- Engineering fundamentals
- Engineering mechanics
- Equipment and machinery
- Forces (type)
- Human and behavioral factors
- Impact forces
- Information management
- Motion (dynamics)
- Practice and Profession
- Probe instruments
- Solid mechanics
- Systems engineering
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.