Robot-Assisted Immersive Kinematic Experience Transfer for Welding Training
Publication: Journal of Computing in Civil Engineering
Volume 37, Issue 2
Abstract
Human motor skills are critical for executing a variety range of tasks in construction. Traditional hands-on training is resource and labor intensive, whereas virtual training, such as video demonstrations, cannot provide trainees with egocentric kinesthetic or proprioceptive experience such as muscular engagement. It is important to develop remote training methods that can provide rich sensory feedback and leverage the trainee’s proprioception. This paper proposes a novel remote motor skill training system that can transfer experts’ kinematic and kinesthetic experience, including both positional and force experience, to novice trainees by using virtual reality (VR) and a robot arm without the physical presence of the experts. The system uses VR to simulate virtual operation scenarios and interactions to provide an immersive operation experience. The robotic system records experts’ kinematic and kinesthetic patterns and trains novices with perceptual learning. The system design was demonstrated with a welding training task. A welding simulator was built with a Unity engine and a seven-degrees-of-freedom robot arm, which provided high-fidelity welding experience and could actively guide welding trainees. It was found that the welding simulator was resilient to external disturbance and provided accurate feedback and guidance. The proposed system contributes to the design of a more embodied remote motor skill training method.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, or code generated or used during the study are available from the corresponding author by request, or the data can be accessed directly at: https://github.com/gilbert-yy/Robot-Assisted-Motor-Learning.git.
Acknowledgments
The authors would like to thank the help from our colleagues Mr. Fang Xu and Ms. Xiwei Lou. This material is supported by the National Science Foundation (NSF) Grant 2024784. Any opinions, findings, conclusions, or recommendations expressed in this article are those of the authors and do not reflect the views of the NSF.
References
3DSystems. 2022. “Touch.” Accessed June 17, 2022. https://www.3dsystems.com/haptics-devices/touch.
Abu-Dakka, F. J., and M. Saveriano. 2020. “Variable impedance control and learning—A review.” Front. Rob. AI 7 (Dec): 590681. https://doi.org/10.3389/frobt.2020.590681.
Adams, J. A., D. Gopher, and G. Lintern. 1975. “The effects of visual and proprioceptive feedback on motor learning.” In Proc., of the Human Factors Society Annual Meeting, 162–165. Thousand Oaks, CA: SAGE.
anarkik3d. 2022. “Falcon haptic device.” Accessed June 17, 2022. https://anarkik3d.co.uk/product/falcon-haptic-device/.
Anderson, D. I., K. R. Lohse, T. C. V. Lopes, and A. M. Williams. 2021. “Individual differences in motor skill learning: Past, present and future.” Hum. Mov. Sci. 78 (Aug): 102818. https://doi.org/10.1016/j.humov.2021.102818.
Aristidou, A., J. Lasenby, Y. Chrysanthou, and A. Shamir. 2018. “Inverse kinematics techniques in computer graphics: A survey.” In Proc., Computer Graphics Forum, 35–58. New York: Wiley.
Asadi, K., V. R. Haritsa, K. Han, and J.-P. Ore. 2021. “Automated object manipulation using vision-based mobile robotic system for construction applications.” J. Comput. Civ. Eng. 35 (1): 04020058. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000946.
AWS (American Welding Society). 2022. “American welding society.” Accessed July 2, 2022. https://www.aws.org/home.
Ayoub, A., and Y. Pulijala. 2019. “The application of virtual reality and augmented reality in oral & maxillofacial surgery.” BMC Oral Health 19 (1): 1–8. https://doi.org/10.1186/s12903-019-0937-8.
Bark, K., E. Hyman, F. Tan, E. Cha, S. A. Jax, L. J. Buxbaum, and K. J. Kuchenbecker. 2015. “Effects of vibrotactile feedback on human learning of arm motions.” IEEE Trans. Neural Syst. Rehabil. Eng. 23 (1): 51–63. https://doi.org/10.1109/TNSRE.2014.2327229.
Beets, I. A., M. Macé, R. L. Meesen, K. Cuypers, O. Levin, and S. P. Swinnen. 2012. “Active versus passive training of a complex bimanual task: Is prescriptive proprioceptive information sufficient for inducing motor learning?” PLoS One 7 (5): e37687. https://doi.org/10.1371/journal.pone.0037687.
Bond, M. 2021. “Schools and emergency remote education during the COVID-19 pandemic: A living rapid systematic review.” Asian J. Distance Educ. 15 (2): 191–247. https://doi.org/10.5281/zenodo.4425683.
Brosque, C., E. G. Herrero, Y. Chen, R. Joshi, O. Khatib, and M. Fischer. 2021. “Collaborative welding and joint sealing robots with haptic feedback.” In Proc., Int. Symp. on Automation and Robotics in Construction (ISARC), 1–8. New York: International Association for Automation and Robotics in Construction.
BSI (British Standards Institution). 2011. “ISO 10218-1:2011 Robots and robotic devices—Safety requirements for industrial robots—Part 1: Robots.” Accessed October 7, 2022. https://www.iso.org/standard/51330.html.
BSI (British Standards Institution). 2016. “ISO/TS 15066:2016 Robots and robotic devices—Collaborative robots.” Accessed October 7, 2022. https://www.iso.org/standard/62996.html.
Buss, S. R. 2004. “Introduction to inverse kinematics with Jacobian transpose, pseudoinverse and damped least squares methods.” IEEE J. Rob. Autom. 17 (1–19): 16.
Carlson, A. G., E. Rowe, and T. W. Curby. 2013. “Disentangling fine motor skills’ relations to academic achievement: The relative contributions of visual-spatial integration and visual-motor coordination.” J. Genet. Psychol. 174 (5–6): 514–533. https://doi.org/10.1080/00221325.2012.717122.
Charness, G., U. Gneezy, and M. A. Kuhn. 2012. “Experimental methods: Between-subject and within-subject design.” J. Econ. Behav. Organ. 81 (1): 1–8. https://doi.org/10.1016/j.jebo.2011.08.009.
Chen, J., and Y. K. Cho. 2022. “Rapid scan-to-building information modeling using robotics and artificial intelligence for construction applications.” In Research companion to building information modeling, 212–229. Cheltenham, UK: Edward Elgar Publishing.
Chen, L. H., Y. H. Kee, Y.-H. Hung, and S.-H. Lin. 2016. “Improving motor performance during initial skills acquisition through regulatory fit: An experimental study based on ball throwing task and small financial reward.” Curr. Psychol. 35 (3): 403–409. https://doi.org/10.1007/s12144-015-9307-0.
Chiyohara, S., J.-I. Furukawa, T. Noda, J. Morimoto, and H. Imamizu. 2020. “Passive training with upper extremity exoskeleton robot affects proprioceptive acuity and performance of motor learning.” Sci. Rep. 10 (1): 1–9. https://doi.org/10.1038/s41598-020-68711-x.
Clarke, S., T. Rhodes, C. G. Atkeson, and O. Kroemer. 2018. “Learning audio feedback for estimating amount and flow of granular material.” In Vol. 87 of Proc., 2nd Conf., on Robot Learning, 529–550. New York: Proceedings of Machine Learning Research.
Crick, C., G. Jay, S. Osentoski, B. Pitzer, and O. C. Jenkins. 2017. “Rosbridge: ROS for non-ROS users.” In Robotics research, 493–504. Boston: Springer.
D’Agostino, R., and E. S. Pearson. 1973. “Tests for departure from normality. Empirical results for the distributions of b2 and√ b1.” Biometrika 60 (3): 613–622. https://doi.org/10.2307/2335012.
Darainy, M., S. Vahdat, and D. J. Ostry. 2013. “Perceptual learning in sensorimotor adaptation.” J. Neurophysiol. 110 (9): 2152–2162. https://doi.org/10.1152/jn.00439.2013.
Donovan, J. J., and D. J. Radosevich. 1999. “A meta-analytic review of the distribution of practice effect: Now you see it, now you don’t.” J. Appl. Psychol. 84 (5): 795. https://doi.org/10.1037/0021-9010.84.5.795.
Eck, U., F. Pankratz, C. Sandor, G. Klinker, and H. Laga. 2015. “Precise haptic device co-location for visuo-haptic augmented reality.” IEEE Trans. Visual Comput. Graphics 21 (12): 1427–1441. https://doi.org/10.1109/TVCG.2015.2480087.
Faria, A. L., M. S. Cameirão, J. F. Couras, J. R. O. Aguiar, G. M. Costa, and S. Bermúdez i Badia. 2018. “Combined cognitive-motor rehabilitation in virtual reality improves motor outcomes in chronic stroke—A pilot study.” Front. Psychol. 9 (May): 854. https://doi.org/10.3389/fpsyg.2018.00854.
Felix, C. V. 2020. “The role of the teacher and AI in education.” In International perspectives on the role of technology in humanizing higher education. Bingley, UK: Emerald Publishing.
Flament, D., J. M. Ellermann, S. G. Kim, K. Uǧurbil, and T. J. Ebner. 1996. “Functional magnetic resonance imaging of cerebellar activation during the learning of a visuomotor dissociation task.” Hum. Brain Mapp. 4 (3): 210–226. https://doi.org/10.1002/hbm.460040302.
FrankaEmika. 2022. “Franka Emika robot arm.” Accessed July 4, 2022. https://www.franka.de/.
Gallegos-Nieto, E., H. I. Medellín-Castillo, G. González-Badillo, and T. Lim. 2014. “Virtual training of assembly tasks using virtual reality techniques and haptic systems.” In Proc., ASME 2014 Int. Mechanical Engineering Congress and Exposition. New York: ASME. https://doi.org/10.1115/IMECE2014-39270.
GitHub. 2019. “ROS# GitHub page.” Accessed June 28, 2022. https://github.com/siemens/ros-sharp.
Habibnezhad, M., S. Shayesteh, H. Jebelli, J. Puckett, and T. Stentz. 2021. “Comparison of ironworker’s fall risk assessment systems using an immersive biofeedback simulator.” Autom. Constr. 122 (Feb): 103471. https://doi.org/10.1016/j.autcon.2020.103471.
Hirano, M., M. Sakurada, and S. Furuya. 2020. “Overcoming the ceiling effects of experts’ motor expertise through active haptic training.” Sci. Adv. 6 (47): eabd2558. https://doi.org/10.1126/sciadv.abd2558.
Hogan, N. 1984. “Impedance control: An approach to manipulation.” In Proc., 1984 American Control Conf., 304–313. New York: IEEE.
Hussein, A., F. García, and C. Olaverri-Monreal. 2018. “ROS and unity based framework for intelligent vehicles control and simulation.” In Proc., 2018 IEEE Int. Conf. on Vehicular Electronics and Safety (ICVES), 1–6. New York: IEEE.
Ipsita, A., L. Erickson, Y. Dong, J. Huang, A. K. Bushinski, S. Saradhi, A. M. Villanueva, K. A. Peppler, T. S. Redick, and K. Ramani. 2022. “Towards modeling of virtual reality welding simulators to promote accessible and scalable training.” In Proc., CHI Conf. on Human Factors in Computing Systems, 1–21. New York: Association for Computing Machinery.
Jenkins, I. H., D. J. Brooks, P. D. Nixon, R. S. Frackowiak, and R. E. Passingham. 1994. “Motor sequence learning: A study with positron emission tomography.” J. Neurosci. 14 (6): 3775–3790. https://doi.org/10.1523/JNEUROSCI.14-06-03775.1994.
Jeong, S.-Y., I.-J. Choi, Y.-J. Kim, Y.-M. Shin, J.-H. Han, G.-H. Jung, and K.-G. Kim. 2017. “A study on ROS vulnerabilities and countermeasure.” In Proc., Companion of the 2017 ACM/IEEE Int. Conf. on Human-Robot Interaction, 147–148. New York: Association for Computing Machinery.
Kim, H., D. Kim, and M. Kim. 2018. “Mesh-based marching cubes on the GPU.” J. Korea Comput. Graphics Soc. 24 (1): 1–8.
Kim, J., S. You, S. Lee, V. Kamat, and L. Robert. 2015. “Evaluation of human robot collaboration in masonry work using immersive virtual environments.” In Proc., 2015 Int. Conf. on Construction Applications of Virtual Reality. Seoul: Chung-Ang Univ.
Koo, Y., and S. Kim. 2019. “Distributed logging system for ROS-based systems.” In Proc., 2019 IEEE Int. Conf. on Big Data and Smart Computing (BigComp), 1–3. New York: IEEE.
Lakshminarayana, N. 2019. “Large scale multimodal data capture, evaluation and maintenance framework for autonomous driving datasets.” In Proc., IEEE/CVF Int. Conf. on Computer Vision Workshops. New York: IEEE.
Lelevé, A., T. McDaniel, and C. Rossa. 2020. “Haptic training simulation.” Front. Virtual Reality 1 (Jul): 3. https://doi.org/10.3389/frvir.2020.00003.
Levac, D. E., M. E. Huber, and D. Sternad. 2019. “Learning and transfer of complex motor skills in virtual reality: A perspective review.” J. Neuroeng. Rehabil. 16 (1): 1–15. https://doi.org/10.1186/s12984-019-0587-8.
Liang, C.-J., W. McGee, C. C. Menassa, and V. R. Kamat. 2022. “Real-time state synchronization between physical construction robots and process-level digital twins.” Constr. Rob. 6 (1): 57–73. https://doi.org/10.1007/s41693-022-00068-1.
Lieberman, J., and C. Breazeal. 2007. “TIKL: Development of a wearable vibrotactile feedback suit for improved human motor learning.” IEEE Trans. Rob. 23 (5): 919–926. https://doi.org/10.1109/TRO.2007.907481.
Liu, L. Y., Y. Li, and A. Lamontagne. 2018. “The effects of error-augmentation versus error-reduction paradigms in robotic therapy to enhance upper extremity performance and recovery post-stroke: A systematic review.” J. Neuroeng. Rehabil. 15 (1): 1–25. https://doi.org/10.1186/s12984-018-0408-5.
Liu, Y., and H. Jebelli. 2021. “Human-robot co-adaptation in construction: Bio-signal based control of bricklaying robots.” In Proc., Computing in Civil Engineering 2021, edited by R. R. A. Issa, 304–312. Reston, VA: ASCE.
Long, T., R. D. Giacalone, and A. T. Asbeck. 2021. “Directional perception of force in a virtual reality environment.” IEEE Access 9 (Aug): 110346–110358. https://doi.org/10.1109/ACCESS.2021.3102390.
Lorensen, W. E., and H. E. Cline. 1987. “Marching cubes: A high resolution 3D surface construction algorithm.” ACM Siggraph Comput. Graphics 21 (4): 163–169. https://doi.org/10.1145/37402.37422.
Maliha, S. G., J. R. Diaz-Siso, N. M. Plana, A. Torroni, and R. L. Flores. 2018. “Haptic, physical, and web-based simulators: Are they underused in maxillofacial surgery training?” J. Oral Maxillofacial Surg. 76 (11): 2424. https://doi.org/10.1016/j.joms.2018.06.177.
Maxwell, D., E. Oster, S. Lynch, D. Maxwell, E. Oster, and S. Lynch. 2018. “Evaluating the applicability of repurposed entertainment virtual reality devices for military training.” In Vol. 28 of Proc., MODSIM World 2018, 1–10. Norfolk, VA: MODSIM World.
McGregor, H. R., J. G. Cashaback, and P. L. Gribble. 2018. “Somatosensory perceptual training enhances motor learning by observing.” J. Neurophysiol. 120 (6): 3017–3025. https://doi.org/10.1152/jn.00313.2018.
Mercado, V. R., M. Marchal, and A. Lécuyer. 2021. “‘Haptics on-demand’: A survey on encountered-type haptic displays.” IEEE Trans. Haptic 14 (3): 449–464. https://doi.org/10.1109/TOH.2021.3061150.
Miller. 2022. “Miller welds.” Accessed June 3, 2022. https://www.millerwelds.com/resources/article-library/addressing-the-welder-shortage-with-technology.
Montoya, A. K. 2022. “Selecting a within- or between-subject design for mediation: Validity, causality, and statistical power.” Multivar. Behav. Res. 57 (3): 1–21. https://doi.org/10.1080/00273171.2022.2077287.
Mourtzis, D. 2018. “Development of skills and competences in manufacturing towards Education 4.0: A teaching factory approach.” In Proc., Int. Conf. on the Industry 4.0 Model for Advanced Manufacturing, 194–210. Boston: Springer.
Nair, J. 2022. “Motion of electrode (with diagram).” Accessed June 10, 2022. https://www.yourarticlelibrary.com/metallurgy/arc-welding-electrodes/motion-of-electrode-with-diagram/83385#:∼:text=Motion%20of%20electrode%20is%20an%20important%20point%20of,surface%20plate%2C%20%E2%80%98weaving%20motion%E2%80%99%20should%20be%20practiced%20properly.
O’Connor, K. E. 2008. “‘You choose to care’: Teachers, emotions and professional identity.” Teach. Teach. Educ. 24 (1): 117–126. https://doi.org/10.1016/j.tate.2006.11.008.
Papakostas, C., C. Troussas, A. Krouska, and C. Sgouropoulou. 2022. “User acceptance of augmented reality welding simulator in engineering training.” Educ. Inf. Technol. 27 (1): 791–817. https://doi.org/10.1007/s10639-020-10418-7.
Quigley, M., K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, and A. Y. Ng. 2009. “ROS: An open-source robot operating system.” In Proc., ICRA Workshop on Open Source Software, 5. New York: IEEE.
Ranganathan, R., and K. M. Newell. 2010. “Motor learning through induced variability at the task goal and execution redundancy levels.” J. Motor Behav. 42 (5): 307–316. https://doi.org/10.1080/00222895.2010.510542.
Reisdorph, N., R. Stearman, K. Kechris, T. L. Phang, R. Reisdorph, J. Prenni, D. J. Erle, C. Coldren, K. Schey, and A. Nesvizhskii. 2013. “Hands-on workshops as an effective means of learning advanced technologies including genomics, proteomics and bioinformatics.” Genomics Proteomics Bioinf. 11 (6): 368–377. https://doi.org/10.1016/j.gpb.2013.10.002.
Ricca, A., A. Chellali, and S. Otrnane. 2021. “The influence of hand visualization in tool-based motor-skills training, a longitudinal study.” In Proc., 2021 IEEE Virtual Reality and 3D User Interfaces (VR), 103–112. New York: IEEE.
Roozbahani, H., and H. Handroos. 2019. “A novel haptic interface and universal control strategy for International thermonuclear experimental reactor (ITER) welding/machining assembly robot.” Rob. Comput. Integr. Manuf. 57 (Jun): 255–270. https://doi.org/10.1016/j.rcim.2018.12.011.
Rosenkranz, K., and J. C. Rothwell. 2012. “Modulation of proprioceptive integration in the motor cortex shapes human motor learning.” J. Neurosci. 32 (26): 9000–9006. https://doi.org/10.1523/JNEUROSCI.0120-12.2012.
Rusli, F. N., A. Zulkifli, M. Bin Saad, and Y. M. Yussop. 2019. “A study of students’ motivation in using the mobile arc welding learning app.” Int. J. Interact. Mobile Technol. 13 (10): 89–105. https://doi.org/10.3991/ijim.v13i10.11305.
Sakamoto, T., and T. Kondo. 2015. “Visuomotor learning by passive motor experience.” Front. Hum. Neurosci. 9 (May): 279. https://doi.org/10.3389/fnhum.2015.00279.
Schüler, T., L. F. D. Santos, and S. Hoermann. 2015. “Designing virtual environments for motor rehabilitation: Towards a framework for the integration of best-practice information.” In Proc., 2015 Int. Conf. on Virtual Rehabilitation (ICVR), 145–146. Vienna, Austria: Kontakt.
Shankhwar, K., T.-J. Chuang, Y.-Y. Tsai, and S. Smith. 2022. “A visuo-haptic extended reality–based training system for hands-on manual metal arc welding training.” Int. J. Adv. Manuf. Technol. 121 (3–4): 1–17. https://doi.org/10.1007/s00170-022-09328-4.
Siemens. 2019. “Siemens/ROS-sharp.” Accessed February 6, 2020. https://github.com/siemens/ros-sharp.
Sternad, D. 2015. “From theoretical analysis to clinical assessment and intervention: Three interactive motor skills in a virtual environment.” In Proc., 2015 Int. Conf. on Virtual Rehabilitation (ICVR), 265–272. New York: IEEE.
Stillman, B. C. 2002. “Making sense of proprioception: The meaning of proprioception, kinaesthesia and related terms.” Physiotherapy 88 (11): 667–676. https://doi.org/10.1016/S0031-9406(05)60109-5.
Thielbar, K. O., T. J. Lord, H. C. Fischer, E. C. Lazzaro, K. C. Barth, M. E. Stoykov, K. M. Triandafilou, and D. G. Kamper. 2014. “Training finger individuation with a mechatronic-virtual reality system leads to improved fine motor control post-stroke.” J. Neuroeng. Rehabil. 11 (1): 171. https://doi.org/10.1186/1743-0003-11-171.
Unity. 2022. “Triplanar node.” Accessed April 16, 2022. https://docs.unity3d.com/Packages/[email protected]/manual/Triplanar-Node.html#:∼:text=Triplanar%20is%20a%20method%20of,the%20normal%2C%20or%20surface%20angle.
Van Beek, F. E., W. M. B. Tiest, and A. M. Kappers. 2013. “Anisotropy in the haptic perception of force direction and magnitude.” IEEE Trans. Haptic 6 (4): 399–407. https://doi.org/10.1109/TOH.2013.37.
Wang, P., X. Bai, M. Billinghurst, S. Zhang, D. Han, H. Lv, W. He, Y. Yan, X. Zhang, and H. Min. 2019. “An MR remote collaborative platform based on 3D CAD models for training in industry.” In Proc., 2019 IEEE Int. Symp. on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), 91–92. New York: IEEE.
Wang, X., C.-J. Liang, C. C. Menassa, and V. R. Kamat. 2021. “Interactive and immersive process-level digital twin for collaborative human–robot construction work.” J. Comput. Civ. Eng. 35 (6): 04021023. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000988.
Wang, Y., Y. Chen, Z. Nan, and Y. Hu. 2006. “Study on welder training by means of haptic guidance and virtual reality for arc welding.” In Proc., 2006 IEEE Int. Conf. on Robotics and Biomimetics, 954–958. New York: IEEE.
WeldGuru. 2022. “Welding electrodes & filler rods explained.” Accessed April 25, 2022. https://weldguru.com/welding-electrode/.
Weldtube. 2019. “6010 Root Pass—Whip and Pause Technique—Plate/Pipe.” Accessed April 25, 2022. https://www.youtube.com/watch?v=aSoZ3r5eI_U.
Wenk, N., J. Penalver-Andres, K. A. Buetler, T. Nef, R. M. Müri, and L. Marchal-Crespo. 2021. “Effect of immersive visualization technologies on cognitive load, motivation, usability, and embodiment.” Virtual Reality 25 (3): 1–25. https://doi.org/10.1007/s10055-021-00565-8.
White, S. A., M. Prachyabrued, T. L. Chambers, C. W. Borst, and D. Reiners. 2011. “Low-cost simulated MIG welding for advancement in technical training.” Virtual Reality 15 (1): 69–81. https://doi.org/10.1007/s10055-010-0162-x.
Whitney, D., E. Rosen, D. Ullman, E. Phillips, and S. Tellex. 2018. “ROS reality: A virtual reality framework using consumer-grade hardware for ROS-enabled robots.” In Proc., 2018 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), 1–9. New York: IEEE.
Wierinck, E., V. Puttemans, S. Swinnen, and D. van Steenberghe. 2005. “Effect of augmented visual feedback from a virtual reality simulation system on manual dexterity training.” Eur. J. Dent. Educ. 9 (1): 10–16. https://doi.org/10.1111/j.1600-0579.2004.00351.x.
Wilcoxon, F. 1992. “Individual comparisons by ranking methods.” In Breakthroughs in statistics, 196–202. Boston: Springer.
Wong, J. D., D. A. Kistemaker, A. Chin, and P. L. Gribble. 2012. “Can proprioceptive training improve motor learning?” J. Neurophysiol. 108 (12): 3313–3321. https://doi.org/10.1152/jn.00122.2012.
Xia, P., F. Xu, T. Zhou, and J. Du. 2022a. “Benchmarking human versus robot performance in emergency structural inspection.” J. Constr. Eng. Manage. 148 (8): 04022070. https://doi.org/10.1061/(ASCE)CO.1943-7862.0002322.
Xia, P., F. Xu, Q. Zhu, and J. Du. 2022b. “Human robot comparison in rapid structural inspection.” In Proc., Construction Research Congress 2022: Computer Applications, Automation, and Data Analytics, edited by F. Jazizadeh, T. Shealy, and M. J. Garvin, 570–580. Reston, VA: ASCE.
Ye, Y. 2022. “Robot-assisted motor skill training.” Accessed August 12, 2022. https://www.youtube.com/watch?v=MUFVscm5k7g.
Ye, Y., Y. Shi, Y. Lee, G. Burks, D. Srinivasan, and J. Du. 2022a. “Exoskeleton training through haptic sensation transfer in immersive virtual environment.” In Proc., Construction Research Congress 2022: Computer Applications, Automation, and Data Analytics, edited by F. Jazizadeh, T. Shealy, and M. J. Garvin, 560–569. Reston, VA: ASCE.
Ye, Y., Y. Shi, D. Srinivasan, and J. Du. 2022b. “Sensation transfer for immersive exoskeleton motor training: Implications of haptics and viewpoints.” Autom. Constr. 141 (Sep): 104411. https://doi.org/10.1016/j.autcon.2022.104411.
Ye, Y., Y. Shi, P. Xia, J. Kang, O. Tyagi, R. K. Mehta, and J. Du. 2022c. “Cognitive characteristics in firefighter wayfinding tasks: An eye-tracking analysis.” Adv. Eng. Inf. 53 (Aug): 101668. https://doi.org/10.1016/j.aei.2022.101668.
Yem, V., H. Kuzuoka, N. Yamashita, R. Shibusawa, H. Yano, and J. Yamashita. 2012. “Assisting hand skill transfer of tracheal intubation using outer-covering haptic display.” In Proc., SIGCHI Conf. on Human Factors in Computing Systems, 3177–3180. New York: Association for Computing Machinery.
Yoshioka, S., K. Matsuura, and N. Gotoda. 2015. “Hand-motion analysis for development of double-unders skill.” Procedia Comput. Sci. 60 (Jan): 775–783. https://doi.org/10.1016/j.procs.2015.08.236.
You, S., J.-H. Kim, S. Lee, V. Kamat, and L. P. Robert Jr. 2018. “Enhancing perceived safety in human–robot collaborative construction using immersive virtual environments.” Autom. Constr. 96 (Dec): 161–170. https://doi.org/10.1016/j.autcon.2018.09.008.
Yousif, N., J. Cole, J. Rothwell, and J. Diedrichsen. 2015. “Proprioception in motor learning: Lessons from a deafferented subject.” Exp. Brain Res. 233 (8): 2449–2459. https://doi.org/10.1007/s00221-015-4315-8.
Zhou, T., Q. Zhu, W. Dixon, and J. Du. 2020a. “Intuitive telerobotic controls based on real-time digital twins simulation.” Autom. Constr. 19 (1): 128–144. https://doi.org/10.5604/01.3001.0013.0464.
Zhou, T., Q. Zhu, and J. Du. 2020b. “Intuitive robot teleoperation for civil engineering operations with virtual reality and deep learning scene reconstruction.” Adv. Eng. Inf. 46 (Oct): 101170. https://doi.org/10.1016/j.aei.2020.101170.
Zhu, Q., J. Du, Y. Shi, and P. Wei. 2021. “Neurobehavioral assessment of force feedback simulation in industrial robotic teleoperation.” Autom. Constr. 126 (Jun): 103674. https://doi.org/10.1016/j.autcon.2021.103674.
Zhu, Q., T. Zhou, and J. Du. 2022a. “Haptics-based force balance controller for tower crane payload sway controls.” Autom. Constr. 144 (Dec): 104597. https://doi.org/10.1016/j.autcon.2022.104597.
Zhu, Q., T. Zhou, and J. Du. 2022b. “Upper-body haptic system for snake robot teleoperation in pipelines.” Adv. Eng. Inf. 51 (Jan): 101532. https://doi.org/10.1016/j.aei.2022.101532.
Information & Authors
Information
Published In
Journal of Computing in Civil Engineering
Volume 37 • Issue 2 • March 2023
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Aug 13, 2022
Accepted: Nov 16, 2022
Published online: Jan 12, 2023
Published in print: Mar 1, 2023
Discussion open until: Jun 12, 2023
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Junjie Chen, Weisheng Lu, Yipeng Pan, Yonglin Fu, Building “RoboAvatar”: Industry Foundation Classes–Based Digital Representation of Robots in the Built Environment, Journal of Computing in Civil Engineering, 10.1061/JCCEE5.CPENG-5723, 38, 4, (2024).
- Sungboo Yoon, Moonseo Park, Changbum R. Ahn, LaserDex: Improvising Spatial Tasks Using Deictic Gestures and Laser Pointing for Human–Robot Collaboration in Construction, Journal of Computing in Civil Engineering, 10.1061/JCCEE5.CPENG-5715, 38, 3, (2024).
- Yang Ye, Thanakon Uthai, Pengxiang Xia, Tianyu Zhou, Jing Du, User Experience and Workload Evaluation in Robot-Assisted Virtual Reality Welding Training, Construction Research Congress 2024, 10.1061/9780784485293.011, (99-108), (2024).
- Fang Xu, Tianyu Zhou, Yang Ye, Jing Du, Augmented Telepresence: Enhancing Robot Arm Control with Mixed Reality for Dexterous Manipulation, Construction Research Congress 2024, 10.1061/9780784485262.074, (727-738), (2024).
- Jiahao Wu, Yang Ye, Jing Du, Multi-Objective Reinforcement Learning for Autonomous Drone Navigation in Urban Area, Construction Research Congress 2024, 10.1061/9780784485262.072, (707-716), (2024).
- Yang Ye, Fang Xu, Jing Du, Collaborative Virtual Training with Embodied Physics and Haptic Feedback: Construction Manual Material Handling as an Example, Computing in Civil Engineering 2023, 10.1061/9780784485231.005, (36-44), (2024).
- Yang Ye, Pengxiang Xia, Tianyu Zhou, Jing Du, Spatial Memory of BIM and Virtual Reality: Mental Mapping Study, Journal of Construction Engineering and Management, 10.1061/JCEMD4.COENG-12808, 149, 7, (2023).