Collaborative Virtual Training with Embodied Physics and Haptic Feedback: Construction Manual Material Handling as an Example
Publication: Computing in Civil Engineering 2023
ABSTRACT
Collaborative motor tasks involving motion corporations among different human operators are commonly seen in many working scenarios, such as surgery operation, fabrication assembly, health care, and material handling. Although virtual training for these tasks has been discussed along with the advancement of communication technologies, few studies have examined the technical procedures and theoretical background of integrating embodied physics and haptic feedback with the virtual training of collaborative motor tasks. In this study, we choose collaborative material handling as an example to implement embodied physics in virtual training for collaborative motor tasks. Multi-player virtual reality (VR) environment is developed to establish a real-time training environment between geographically distributed operators. The virtual materials and environment are synchronized via a cloud server, creating a homogeneous context for the collaboration task. Geographically distributed operators can interact with the virtual environment by controlling a whole-body tracked virtual operator to report or move hazards and complete the material handling task as a team. In addition, the feeling of physical contact can be enhanced with haptic feedback. A pilot test shows that the collaborative training environment enhances safety awareness and degree of engagement. Meanwhile, simulated embodied physics can effectively bridge the gap between virtual and real training scenarios and enhance the task-learning experience.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Bark, K., Hyman, E., Tan, F., Cha, E., Jax, S. A., Buxbaum, L. J., and Kuchenbecker, K. J. (2015). “Effects of vibrotactile feedback on human learning of arm motions.” IEEE Trans Neural Syst Rehabil Eng, 23(1), 51–63.
Burdea, G., Richard, P., and Coiffet, P. (1996). “Multimodal virtual reality: Input‐output devices, system integration, and human factors.” International Journal of Human‐Computer Interaction, 8(1), 5–24.
Caserman, P., Schmidt, P., Göbel, T., Zinnäcker, J., Kecke, A., and Göbel, S. (2022). “Impact of Full-Body Avatars in Immersive Multiplayer Virtual Reality Training for Police Forces.” IEEE Transactions on Games, 14(4), 706–714.
Clemes, S. A., Haslam, C. O., and Haslam, R. A. (2010). “What constitutes effective manual handling training? A systematic review.” Occupational medicine, 60(2), 101–107.
Denis, D., Gonella, M., Comeau, M., and Lauzier, M. (2020). “Questioning the value of manual material handling training: A scoping and critical literature review.” Applied Ergonomics, 89, 103186.
Endsley, M. R., Selcon, S. J., Hardiman, T. D., and Croft, D. G. “A comparative analysis of SAGAT and SART for evaluations of situation awareness.” Proc., Proceedings of the human factors and ergonomics society annual meeting, SAGE Publications Sage CA: Los Angeles, CA, 82–86.
Hayes, J., Dwivedi, S., Karthikeyan, R., Abujelala, M., Kang, J., Ye, Y., Du, E., and Mehta, R. K. “Identifying Early Predictors of Learning in VR-based Drone Training.” Proc., Proceedings of the Human Factors and Ergonomics Society Annual Meeting, SAGE Publications Sage CA: Los Angeles, CA, 1872–1876.
Khanal, P., Vankipuram, A., Ashby, A., Vankipuram, M., Gupta, A., Drumm-Gurnee, D., Josey, K., Tinker, L., and Smith, M. (2014). “Collaborative virtual reality based advanced cardiac life support training simulator using virtual reality principles.” Journal of Biomedical Informatics, 51, 49–59.
Lieberman, J., and Breazeal, C. (2007). “TIKL: Development of a Wearable Vibrotactile Feedback Suit for Improved Human Motor Learning.” IEEE Transactions on Robotics, 23(5), 919–926.
Mourtzis, D. “Development of skills and competences in manufacturing towards Education 4.0: A teaching factory approach.” Proc., International Conference on the Industry 4.0 model for Advanced Manufacturing, Springer, 194–210.
Rajesh, R. (2016). “Manual material handling: A classification scheme.” Procedia Technology, 24, 568–575.
Schüler, T., Santos, L. F. D., and Hoermann, S. “Designing virtual environments for motor rehabilitation: Towards a framework for the integration of best-practice information.” Proc., 2015 International Conference on Virtual Rehabilitation (ICVR), 145–146.
Wang, P., Bai, X., Billinghurst, M., Zhang, S., Han, D., Lv, H., He, W., Yan, Y., Zhang, X., and Min, H. “An MR Remote Collaborative Platform Based on 3D CAD Models for Training in Industry.” Proc., 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), 91–92.
Wang, X., Dong, X. S., Choi, S. D., and Dement, J. (2017). “Work-related musculoskeletal disorders among construction workers in the United States from 1992 to 2014.” Occup Environ Med, 74(5), 374–380.
Witmer, B. G., and Singer, M. J. (1998). “Measuring presence in virtual environments: A presence questionnaire.” Presence, 7(3), 225–240.
Ye, Y., Shi, Y., Lee, Y., Burks, G., Srinivasan, D., and Du, J. (2022). “Exoskeleton Training through Haptic Sensation Transfer in Immersive Virtual Environment.” Construction Research Congress 2022, 560–569.
Ye, Y., Shi, Y., Srinivasan, D., and Du, J. (2022). “Sensation transfer for immersive exoskeleton motor training: Implications of haptics and viewpoints.” Automation in Construction, 141, 104411.
Ye, Y., Shi, Y., Xia, P., Kang, J., Tyagi, O., Mehta, R. K., and Du, J. (2022). “Cognitive characteristics in firefighter wayfinding Tasks: An Eye-Tracking analysis.” Advanced Engineering Informatics, 53, 101668.
Ye, Y., Zhou, T., and Du, J. (2023). “Robot-Assisted Immersive Kinematic Experience Transfer for Welding Training.” Journal of Computing in Civil Engineering, 37(2), 04023002.
Yem, V., Kuzuoka, H., Yamashita, N., Shibusawa, R., Yano, H., and Yamashita, J. (2012). “Assisting hand skill transfer of tracheal intubation using outer-covering haptic display.” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Association for Computing Machinery, 3177–3180.
Information & Authors
Information
Published In
Computing in Civil Engineering 2023
Pages: 36 - 44
History
Published online: Jan 25, 2024
ASCE Technical Topics:
- Business management
- Business organizations
- Construction materials
- Continuum mechanics
- Corporations
- Disaster risk management
- Disasters and hazards
- Dynamics (solid mechanics)
- Education
- Engineering fundamentals
- Engineering materials (by type)
- Engineering mechanics
- Fabrication
- Hazardous substances
- Man-made disasters
- Materials engineering
- Materials processing
- Models (by type)
- Motion (dynamics)
- Organizations
- Practice and Profession
- Public administration
- Public health and safety
- Simulation models
- Solid mechanics
- Training
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.