Grip State Recognition for Enabling Safe Human-Robot Object Handover in Physically Collaborative Construction Work
Publication: Computing in Civil Engineering 2023
ABSTRACT
Physical interactions between humans and construction robots must inevitably occur under a capacity-based task allocation scheme for successful implementation of a human-robot collaboration (HRC) based workflow. For example, in staging and installing components, a robot can complete physically challenging material handling tasks and a human worker can then creatively manipulate the materials for installation. When materials are handed over from robots to humans, tandem manipulation of objects and physical interaction is necessary. However, this poses several challenges to human workers’ safety, as the objects might fall or break, if the robot incorrectly perceives the human worker’s grip state. This paper considers object handovers between robots and humans and explores how to build a safe tandem HRC framework by letting the robot imitate the mutual physical state adaptation dynamics inherent in human co-workers. To build such a human physical state-aware robot controller, this paper first proposes the use of a haptic glove-based sensing system to capture the grip strength and gesture of construction workers simultaneously. Secondly, the programming by demonstration (PbD) method is used to automatically program a robot through a one-shot demonstration of natural handover processes without requiring the demonstrators (i.e., construction workers) to have computational or programming expertise. The proposed method outperforms prior robot-to-human object handover studies in handling eight construction materials of random shape, dimensions, and weight distribution. In addition, to enable such construction HRC implementations to readily comply with global safety standards, the proposed method is implemented to adhere to ISO 15066:2016 guidelines.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Brosque, C., Galbally, E., Khatib, O., and Fischer, M. (2020, June). Human-robot collaboration in construction: Opportunities and challenges. In 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA) (pp. 1–8). IEEE.
Chan, W. P., Parker, C. A., Van der Loos, H. M., and Croft, E. A. (2012, March). Grip forces and load forces in handovers: implications for designing human-robot handover controllers. In Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction (pp. 9–16).
Controzzi, M., Singh, H., Cini, F., Cecchini, T., Wing, A., and Cipriani, C. (2018). Humans adjust their grip force when passing an object according to the observed speed of the partner’s reaching out movement. Experimental brain research, 236, 3363–3377.
Deng, M., Wang, X., Li, D., and Menassa, C. C. (2022). Digital ID framework for human-centric monitoring and control of smart buildings. In Building Simulation (Vol. 15, No. 10, pp. 1709–1728). Beijing: Tsinghua University Press.
Döhring, F. R., Müller, H., and Joch, M. (2020). Grip-force modulation in human-to-human object handovers: effects of sensory and kinematic manipulations. Scientific Reports, 10(1), 22381.
Eguiluz, A. G., Rañó, I., Coleman, S. A., and McGinnity, T. M. (2017, May). Reliable object handover through tactile force sensing and effort control in the shadow robot hand. In 2017 IEEE International Conference on Robotics and Automation (ICRA) (pp. 372–377). IEEE.
Fang, B., Jia, S., Guo, D., Xu, M., Wen, S., and Sun, F. (2019). Survey of imitation learning for robotic manipulation. International Journal of Intelligent Robotics and Applications, 3, 362–369.
Feng, C., Xiao, Y., Willette, A., McGee, W., and Kamat, V. R. (2015). Vision guided autonomous robotic assembly and as-built scanning on unstructured construction sites. Automation in Construction, 59, 128–138.
Fitts, P. M. (1951). Human engineering for an effective air-navigation and traffic-control system.
Gehler, P., and Nowozin, S. (2009, September). On feature combination for multiclass object classification. In 2009 IEEE 12th International Conference on Computer Vision (pp. 221–228). IEEE.
Grigore, E. C., Eder, K., Pipe, A. G., Melhuish, C., and Leonards, U. (2013, November). Joint action understanding improves robot-to-human object handover. In 2013 IEEE/RSJ international conference on intelligent robots and systems (pp. 4622–4629). IEEE.
Huang, C. M., Cakmak, M., and Mutlu, B. (2015, July). Adaptive Coordination Strategies for Human-Robot Handovers. In Robotics: science and systems (Vol. 11, pp. 1–10).
Koene, A., Endo, S., Remazeilles, A., Prada, M., and Wing, A. M. (2014, August). Experimental testing of the coglaboration prototype system for fluent human-robot object handover interactions. In The 23rd IEEE International Symposium on Robot and Human Interactive Communication (pp. 249–254). IEEE.
Lee, K. S., and Jung, M. C. (2014). Common patterns of voluntary grasp types according to object shape, size, and direction. International Journal of Industrial Ergonomics, 44(5), 761–768.
Liang, C. J., Wang, X., Kamat, V. R., and Menassa, C. C. (2021). Human–robot collaboration in construction: classification and research trends. Journal of Construction Engineering and Management, 147(10), 03121006.
Lu, Y., Zheng, H., Chand, S., Xia, W., Liu, Z., Xu, X., and Bao, J. (2022). Outlook on human-centric manufacturing towards Industry 5.0. Journal of Manufacturing Systems, 62, 612–627.
Lundeen, K. M., Kamat, V. R., Menassa, C. C., and McGee, W. (2019). Autonomous motion planning and task execution in geometrically adaptive robotized construction work. Automation in Construction, 100, 24–45.
Palinko, O., Rea, F., Sandini, G., and Sciutti, A. (2016, October). Robot reading human gaze: Why eye tracking is better than head tracking for human-robot collaboration. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 5048–5054). IEEE.
Pan, M. K., Knoop, E., Bächer, M., and Niemeyer, G. (2019, November). Fast handovers with a robot character: Small sensorimotor delays improve perceived qualities. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 6735–6741). IEEE.
Park, S., Yu, H., Menassa, C. C., and Kamat, V. R. (2023). A Comprehensive Evaluation of Factors Influencing Acceptance of Robotic Assistants in Field Construction Work. Journal of Management in Engineering, 39(3), 04023010.
Paulino, L., Hannum, C., Varde, A. S., and Conti, C. J. (2022). Search methods in motion planning for mobile robots. In Intelligent Systems and Applications: Proceedings of the 2021 Intelligent Systems Conference (IntelliSys) Volume 3 (pp. 802–822). Springer International Publishing.
Paulun, V. C., Gegenfurtner, K. R., Goodale, M. A., and Fleming, R. W. (2016). Effects of material properties and object orientation on precision grip kinematics. Experimental brain research, 234, 2253–2265.
Roweis, S. (1997). EM algorithms for PCA and SPCA. Advances in neural information processing systems, 10.
Singh, H., Controzzi, M., Cipriani, C., Di Caterina, G., Petropoulakis, L., and Soraghan, J. (2018, September). Online prediction of robot to human handover events using vibrations. In 2018 26th European Signal Processing Conference (EUSIPCO) (pp. 687–691). IEEE.
Thorpe, S., Fize, D., and Marlot, C. (1996). Speed of processing in the human visual system. nature, 381(6582), 520–522.
Wang, X., Liang, C. J., Menassa, C., and Kamat, V. (2020, October). Real-time process-level digital twin for collaborative human-robot construction work. In Proceedings of the International Symposium on Automation and Robotics in Construction (IAARC).
Wang, X., Liang, C. J., Menassa, C. C., and Kamat, V. R. (2021a). Interactive and immersive process-level digital twin for collaborative human–robot construction work. Journal of Computing in Civil Engineering, 35(6), 04021023.
Wang, W., Li, R., Chen, Y., Sun, Y., and Jia, Y. (2021b). Predicting human intentions in human–robot hand-over tasks through multimodal learning. IEEE Transactions on Automation Science and Engineering, 19(3), 2339–2353.
Werremeyer, M. M., and Cole, K. J. (1997). Wrist action affects precision grip force. Journal of Neurophysiology, 78(1), 271–280.
Yu, H., Kamat, V. R., Menassa, C. C., McGee, W., Guo, Y., and Lee, H. (2023). Mutual physical state-aware object handover in full-contact collaborative human-robot construction work. Automation in Construction, 150, 104829.
Information & Authors
Information
Published In
Computing in Civil Engineering 2023
Pages: 787 - 795
History
Published online: Jan 25, 2024
ASCE Technical Topics:
- Automation and robotics
- Business management
- Computer programming
- Computing in civil engineering
- Construction engineering
- Construction management
- Construction methods
- Employment
- Engineering fundamentals
- Human and behavioral factors
- Labor
- Occupational safety
- Personnel management
- Practice and Profession
- Public administration
- Public health and safety
- Safety
- Systems engineering
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.