Real-Time Muscle-Level Haptic Feedback for Enhanced Welding Learning: An sEMG-Based Approach
Publication: Journal of Construction Engineering and Management
Volume 150, Issue 12
Abstract
Engaging muscle groups in proper order is an essential skill for human motor tasks such as welding, contributing to musculoskeletal health and task performance. However, learning the correct muscle engagement strategy for a dedicated human motion task like welding can be challenging due to the difficulty of acquiring the implicit information and carrying it over to novice trainees, which is becoming the bottleneck of workforce training. This paper proposes to monitor and model the implicit muscle engagement strategies using surface electromyographic (sEMG) sensors and provide real-time feedback to novice trainees via vibrotactile devices according to the muscle engagement models. The differences between expert trainers’ and novice trainees’ muscle engagement strategies are processed to control the vibrotactile patterns and magnitudes. A human-subject experiment () was performed to validate the system design in welding training. Our results, which demonstrated the effectiveness of the proposed method in capturing motor control patterns and improving motor skill learning, provide a strong foundation for its application in real-world training scenarios. Furthermore, it was found that providing haptic feedback according to muscle engagement was more effective than visual feedback, especially in force control. It was also noticed that haptic feedback could alleviate the reliance on external feedback compared with visual feedback. This paper presented a novel method and its implementation for welding training, contributing to innovative training methods for the workforce and beyond.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
The authors would like to thank all the experiment participants for joining this study and contributing their thoughts. This material is supported by the National Science Foundation (NSF) Grant No. 2024784. Any opinions, findings, conclusions, or recommendations expressed in this article are those of the authors and do not reflect the views of the NSF.
References
AWS (American Welding Society). 2022a. “American Welding Society.” Accessed November 30, 2022. https://www.aws.org/home.
AWS (American Welding Society). 2022b. “Keyhole welding.” Accessed November 20, 2022. https://awo.aws.org/glossary/keyhole-welding/#:∼:text=A%20technique%20in%20which%20a,to%20form%20the%20weld%20bead.
Bark, K., E. Hyman, F. Tan, E. Cha, S. A. Jax, L. J. Buxbaum, and K. J. Kuchenbecker. 2015. “Effects of vibrotactile feedback on human learning of arm motions.” IEEE Trans. Neural Syst. Rehabil. Eng. 23 (1): 51–63. https://doi.org/10.1109/TNSRE.2014.2327229.
bHaptics. 2022. “bHaptics product manual.” Accessed March 17, 2023. https://www.bhaptics.com.
Blake, O. M., Y. Champoux, and J. M. Wakeling. 2012. “Muscle coordination patterns for efficient cycling.” Med. Sci. Sports Exercise 44 (5): 926–938. https://doi.org/10.1249/MSS.0b013e3182404d4b.
Branscheidt, M., P. Kassavetis, M. Anaya, D. Rogers, H. D. Huang, M. A. Lindquist, and P. Celnik. 2019. “Fatigue induces long-lasting detrimental changes in motor-skill learning.” Elife 8 (Mar): e40578. https://doi.org/10.7554/eLife.40578.
Burdea, G., P. Richard, and P. Coiffet. 1996. “Multimodal virtual reality: Input-output devices, system integration, and human factors.” Int. J. Human-Comput. Interact. 8 (1): 5–24. https://doi.org/10.1080/10447319609526138.
Charness, G., U. Gneezy, and M. A. Kuhn. 2012. “Experimental methods: Between-subject and within-subject design.” J. Econ. Behav. Organ. 81 (1): 1–8. https://doi.org/10.1016/j.jebo.2011.08.009.
Chiyohara, S., J.-I. Furukawa, T. Morimoto, and H. Imamizu. 2020. “Passive training with upper extremity exoskeleton robot affects proprioceptive acuity and performance of motor learning.” Sci. Rep. 10 (1): 1–9. https://doi.org/10.1038/s41598-020-68711-x.
Day, S. 2002. Important factors in surface EMG measurement, 1–17. London: Bortec Biomedical.
Farina, D., R. Merletti, and R. M. Enoka. 2004. “The extraction of neural strategies from the surface EMG.” J. Appl. Physiol. 96 (4): 1486–1495. https://doi.org/10.1152/japplphysiol.01070.2003.
Hadinejad, A., R. Fesharakifard, S. M. Rezaei, and A. Nahvi. 2021. “Virtual manual arc welding by real-time process simulation for an effective training.” AUT J. Mech. Eng. 5 (1): 121–140. https://doi.org/10.22060/ajme.2020.17800.5875.
Haibach-Beach, P. S., M. Perreault, A. Brian, and D. H. Collier. 2023. Motor learning and development. Champaign, IL: Human Kinetics.
Hart, S. G. 2006. “NASA-task load index (NASA-TLX); 20 years later.” In Proc., Human Factors and Ergonomics Society Annual Meeting, 904–908. Los Angeles: SAGE.
Hawkes, D. H., O. A. Khaiyat, A. J. Howard, G. J. Kemp, and S. P. Frostick. 2019. “Patterns of muscle coordination during dynamic glenohumeral joint elevation: An EMG study.” PLoS One 14 (2): e0211800. https://doi.org/10.1371/journal.pone.0211800.
Henriques, D. Y., M. Flanders, and J. F. Soechting. 2004. “Haptic synthesis of shapes and sequences.” J. Neurophysiol. 91 (4): 1808–1821. https://doi.org/10.1152/jn.00998.2003.
Heuer, H., and J. Lüttgen. 2014. “Motor learning with fading and growing haptic guidance.” Exp. Brain Res. 232 (Jun): 2229–2242. https://doi.org/10.1007/s00221-014-3914-0.
Hirano, M., M. Sakurada, and S. Furuya. 2020. “Overcoming the ceiling effects of experts’ motor expertise through active haptic training.” Sci. Adv. 6 (47): eabd2558. https://doi.org/10.1126/sciadv.abd2558.
Hug, F. 2011. “Can muscle coordination be precisely studied by surface electromyography?” J. Electromyogr. Kinesiol. 21 (1): 1–12. https://doi.org/10.1016/j.jelekin.2010.08.009.
Hug, F., J. M. Drouet, Y. Champoux, A. Couturier, and S. Dorel. 2008. “Interindividual variability of electromyographic patterns and pedal force profiles in trained cyclists.” Eur. J. Appl. Physiol. 104 (4): 667–678. https://doi.org/10.1007/s00421-008-0810-y.
Ingram, T. G., S. N. Kraeutner, J. P. Solomon, D. A. Westwood, and S. G. Boe. 2016. “Skill acquisition via motor imagery relies on both motor and perceptual learning.” Behav. Neurosci. 130 (2): 252. https://doi.org/10.1037/bne0000126.
Ipsita, A., et al. 2022. “Towards modeling of virtual reality welding simulators to promote accessible and scalable training.” In Proc., CHI Conf. on Human Factors in Computing Systems, 1–21. New York: Association for Computing Machinery Digital Library.
Kahn, L. E., P. S. Lum, W. Z. Rymer, and D. J. Reinkensmeyer. 2014. “Robot-assisted movement training for the stroke-impaired arm: Does it matter what the robot does?” J. Rehabil. Res. Dev. 43 (5): 619–630. https://doi.org/10.1682/JRRD.2005.03.0056.
Karni, A. 1996. “The acquisition of perceptual and motor skills: A memory system in the adult human cortex.” In Cognitive brain research. Amsterdam, Netherlands: Elsevier.
Kim, K., and J. E. Colgate. 2012. “Haptic feedback enhances grip force control of sEMG-controlled prosthetic hands in targeted reinnervation amputees.” IEEE Trans. Neural Syst. Rehabil. Eng. 20 (6): 798–805. https://doi.org/10.1109/TNSRE.2012.2206080.
Klepsch, M., F. Schmitz, and T. Seufert. 2017. “Development and validation of two instruments measuring intrinsic, extraneous, and germane cognitive load.” Front. Psychol. 8 (5): 1997. https://doi.org/10.3389/fpsyg.2017.01997.
Lathan, C. E., and M. Tracey. 2002. “The effects of operator spatial perception and sensory feedback on human-robot teleoperation performance.” Presence: Teleoperators Virtual Environ. 11 (4): 368–377. https://doi.org/10.1162/105474602760204282.
Lelevé, A., T. McDaniel, and C. Rossa. 2020. “Haptic training simulation.” Front. Virtual Reality 1 (Jun): 3. https://doi.org/10.3389/frvir.2020.00003.
Lenzi, T., S. M. M. De Rossi, N. Vitiello, and M. C. Carrozza. 2012. “Intention-based EMG control for powered exoskeletons.” IEEE Trans. Biomed. Eng. 59 (8): 2180–2190. https://doi.org/10.1109/TBME.2012.2198821.
Luettgen, J., and H. Heuer. 2013. “The influence of robotic guidance on different types of motor timing.” J. Mot. Behav. 45 (3): 249–258. https://doi.org/10.1080/00222895.2013.785926.
Luft, A. R., and M. M. Buitrago. 2005. “Stages of motor skill learning.” Mol. Neurobiol. 32 (3): 205–216. https://doi.org/10.1385/MN:32:3:205.
Luft, A. R., M. M. Buitrago, A. Kaelin-Lang, J. Dichgans, and J. B. Schulz. 2004. “Protein synthesis inhibition blocks consolidation of an acrobatic motor skill.” Learn Mem 11 (4): 379–382. https://doi.org/10.1101/lm.72604.
Mesin, L., R. Merletti, and A. Rainoldi. 2009. “Surface EMG: The issue of electrode location.” J. Electromyogr. Kinesiol. 19 (5): 719–726. https://doi.org/10.1016/j.jelekin.2008.07.006.
Nazelskis, E. 2015. “Identification of the demand of welders in the labor market of Lithuania.” Profesinis Rengimas: Tyrimai ir Realijos 26 (1): 59–79. https://doi.org/10.1515/vtrr-2015-0004.
Panahi, A. K., S. Cho, and C. Gordon. 2021. “Muscle fatigue analysis during welding tasks using sEMG and recurrence quantification analysis.” Int. J. Appl. Indus. Eng. 8 (1): 1–16. https://doi.org/10.4018/IJAIE.287609.
Papakostas, C., C. Troussas, A. Krouska, and C. Sgouropoulou. 2022. “User acceptance of augmented reality welding simulator in engineering training.” Educ. Inf. Technol. 27 (1): 791–817. https://doi.org/10.1007/s10639-020-10418-7.
Parajuli, N., N. Sreenivasan, P. Bifulco, M. Cesarelli, S. Savino, V. Niola, D. Esposito, T. J. Hamilton, G. R. Naik, and U. Gunawardana. 2019. “Real-time EMG based pattern recognition control for hand prostheses: A review on existing methods, challenges and future implementation.” Sensors 19 (20): 4596. https://doi.org/10.3390/s19204596.
PluxBiosignals. 2024. “Bitalino: Learning and exploring biosignals everywhere.” Accessed February 17, 2024. https://www.pluxbiosignals.com/pages/bitalino.
Pointer, M., R. Naylor, J. Warden, G. Senek, C. Shirley, L. Lefcourt, J. Munson, and A. Johnson. 2005. State skill standards: Welding office of career, technical, and adult education. Las Vegas: Nevada Department of Education.
Price, A., M. Kuttolamadom, and S. Obeidat. 2019. “Using virtual reality welding to improve manufacturing process education.” In Proc., 2019 CIEC. New York: ASEE.
Reisdorph, N., R. Stearman, K. Kechris, T. L. Phang, R. Reisdorph, J. Prenni, D. J. Erle, C. Coldren, K. Schey, and A. Nesvizhskii. 2013. “Hands-on workshops as an effective means of learning advanced technologies including genomics proteomics and bioinformatics.” Genomics Proteomics Bioinf. 11 (6): 368–377. https://doi.org/10.1016/j.gpb.2013.10.002.
Ricca, A., A. Chellali, and S. Otrnane. 2021. “The influence of hand visualization in tool-based motor-skills training, A longitudinal study.” In Proc., 2021 IEEE Virtual Reality and 3D User Interfaces (VR), 103–112. New York: IEEE.
Rusli, F. N., A. Zulkifli, M. Bin Saad, and Y. M. Yussop. 2019. A study of students’ motivation in using the mobile arc welding learning app. New York: International Association of Online Engineering.
Sammon, J., A. Perry, L. Beaule, T. Kinkead, D. Clark, and M. Hansen. 2010. “Robot-assisted radical prostatectomy: Learning rate analysis as an objective measure of the acquisition of surgical skill.” BJU Int. 106 (6): 855–860. https://doi.org/10.1111/j.1464-410X.2009.09187.x.
Samuel, O. W., M. G. Asogbon, Y. Geng, A. H. Al-Timemy, S. Pirbhulal, N. Ji, S. Chen, P. Fang, and G. Li. 2019. “Intelligent EMG pattern recognition control method for upper-limb multifunctional prostheses: Advances, current challenges, and future prospects.” IEEE Access 7 (Jun): 10150–10165. https://doi.org/10.1109/ACCESS.2019.2891350.
Schmidt, R. A. 1991. “Frequent augmented feedback can degrade learning: Evidence and interpretations.” In Tutorials in motor neuroscience, 59–75. Berlin: Springer.
Schmidt, R. A., T. D. Lee, C. Winstein, G. Wulf, and H. N. Zelaznik. 2018. Motor control and learning: A behavioral emphasis. Champaign, IL: Human Kinetics.
Schunk, D. H., and B. J. Zimmerman. 1997. “Social origins of self-regulatory competence.” Educ. Psychol. 32 (4): 195–208. https://doi.org/10.1207/s15326985ep3204_1.
Seniam. 2024. “Recommendations for sensor locations in arm or hand muscles.” Accessed February 4, 2024. http://www.seniam.org/.
Shan, G., P. Visentin, G. Tardif, A. Robinson, J. Evans, and S. Crowe. 2012. “Quantifying compensation strategies between gross and fine motor control in violin performance.” Arts Biomech. 1 (2): 143.
Shankhwar, K., T.-J. Chuang, Y.-Y. Tsai, and S. Smith. 2022. “A visuo-haptic extended reality–based training system for hands-on manual metal arc welding training.” Int. J. Adv. Manuf. Technol. 121 (1): 1–17. https://doi.org/10.1007/s00170-022-09328-4.
Siedentop, D. 1976. Developing teaching skills in physical education. Mountain View, CA: Mayfield Pub Co.
Society, A. W. 2024. “Shining a light on the welding workforce.” Accessed April 15, 2024. https://weldingworkforcedata.com/.
Stone, R., E. McLaurin, P. Zhong, and K. Watts. 2013. Full virtual reality vs. integrated virtual reality training in welding. Doral, FL: American Welding Society.
Sun, Z., M. Zhu, X. Shan, and C. Lee. 2022. “Augmented tactile-perception and haptic-feedback rings as human-machine interfaces aiming for immersive interactions.” Nat. Commun. 13 (1): 5224. https://doi.org/10.1038/s41467-022-32745-8.
Torres, E. B. 2011. “Two classes of movements in motor control.” Exp. Brain Res. 215 (Apr): 269–283. https://doi.org/10.1007/s00221-011-2892-8.
Turner, R. S., S. T. Grafton, J. R. Votaw, M. R. Delong, and J. M. Hoffman. 1998. “Motor subcircuits mediating the control of movement velocity: A PET study.” J. Neurophysiol. 80 (4): 2162–2176. https://doi.org/10.1152/jn.1998.80.4.2162.
US Bureau of Labor Statistics. 2024. “Occupational outlook handbook, welders, cutters, solderers, and brazers.” Accessed March 16, 2024. https://www.bls.gov/ooh/production/welders-cutters-solderers-and-brazers.htm.
Vallat, R. 2018. “Pingouin: Statistics in Python.” J. Open Source Software 3 (31): 1026. https://doi.org/10.21105/joss.01026.
Visentin, P., S. Li, G. Tardif, and G. Shan. 2015. “Unraveling mysteries of personal performance style; biomechanics of left-hand position changes (shifting) in violin performance.” PeerJ 3 (Aug): e1299. https://doi.org/10.7717/peerj.1299.
Wan, B., and G. Shan. 2016. “Biomechanical modeling as a practical tool for predicting injury risk related to repetitive muscle lengthening during learning and training of human complex motor skills.” Springerplus 5 (Dec): 1–13. https://doi.org/10.1186/s40064-016-2067-y.
Wang, X., X. S. Dong, S. D. Choi, and J. Dement. 2017. “Work-related musculoskeletal disorders among construction workers in the United States from 1992 to 2014.” Occup. Environ. Med. 74 (5): 374–380. https://doi.org/10.1136/oemed-2016-103943.
Weiss, Y., E. P. Simoncelli, and E. H. Adelson. 2002. “Motion illusions as optimal percepts.” Nat. Neurosci. 5 (6): 598–604. https://doi.org/10.1038/nn0602-858.
WeldGuru. 2022. “Welding electrodes & filler rods explained.” Accessed June 16, 2023. https://weldguru.com/welding-electrode/.
Weldtube. 2019. “6010 root pass—Whip and pause technique—Plate/Pipe.” Accessed October 2, 2023. https://www.youtube.com/watch?v=aSoZ3r5eI_U.
Wierinck, E., V. Puttemans, S. Swinnen, and D. van Steenberghe. 2005. “Effect of augmented visual feedback from a virtual reality simulation system on manual dexterity training.” Eur. J. Dental Educ. 9 (1): 10–16. https://doi.org/10.1111/j.1600-0579.2004.00351.x.
Winstein, C. J., P. S. Pohl, and R. Lewthwaite. 1994. “Effects of physical guidance and knowledge of results on motor learning: Support for the guidance hypothesis.” Res. Q. Exercise Sport 65 (4): 316–323. https://doi.org/10.1080/02701367.1994.10607635.
Wouters, P., H. K. Tabbers, and F. Paas. 2007. “Interactivity in video-based models.” Educ. Psychol. Rev. 19 (3): 327–342. https://doi.org/10.1007/s10648-007-9045-4.
Xie, B., Q. Zhou, and L. Yu. 2015. “A real-time welding training system base on virtual reality.” In Proc., 2015 IEEE Virtual Reality (VR), 309–310. New York: IEEE.
Yang, C., J. Luo, C. Liu, M. Li, and S.-L. Dai. 2018. “Haptics electromyography perception and learning enhanced intelligence for teleoperated robot.” IEEE Trans. Autom. Sci. Eng. 16 (4): 1512–1521. https://doi.org/10.1109/TASE.2018.2874454.
Ye, Y., Y. Shi, Y. Lee, G. Burks, D. Srinivasan, and J. Du. 2022a. “Exoskeleton training through haptic sensation transfer in immersive virtual environment.” In Proc., Construction Research Congress 2022, 560–569. Washington, DC: Human Factors and Ergonomics Society.
Ye, Y., Y. Shi, D. Srinivasan, and J. Du. 2022b. “Sensation transfer for immersive exoskeleton motor training: Implications of haptics and viewpoints.” Autom. Constr. 141 (Sep): 104411. https://doi.org/10.1016/j.autcon.2022.104411.
Ye, Y., T. Zhou, and J. Du. 2023. “Robot-assisted immersive kinematic experience transfer for welding training.” J. Comput. Civ. Eng. 37 (2): 04023002. https://doi.org/10.1061/JCCEE5.CPENG-5138.
Young, A. J., L. H. Smith, E. J. Rouse, and L. J. Hargrove. 2012. “Classification of simultaneous movements using surface EMG pattern recognition.” IEEE Trans. Biomed. Eng. 60 (5): 1250–1258. https://doi.org/10.1109/TBME.2012.2232293.
Zhang, X., G. Shan, Y. Wang, B. Wan, and H. Li. 2019. “Wearables, biomechanical feedback, and human motor-skills’ learning & optimization.” Appl. Sci. 9 (2): 226. https://doi.org/10.3390/app9020226.
Zhong, P., and R. T. Stone. 2013. “Automated kinesthetic trainer enhances kinesthetic memory development.” Int. J. Ind. Ergon. 43 (3): 238–245. https://doi.org/10.1016/j.ergon.2013.02.007.
Zimmerman, B. J., and A. Kitsantas. 2002. “Acquiring writing revision and self-regulatory skill through observation and emulation.” J. Educ. Psychol. 94 (4): 660. https://doi.org/10.1037/0022-0663.94.4.660.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 150 • Issue 12 • December 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Apr 15, 2024
Accepted: Jul 5, 2024
Published online: Sep 30, 2024
Published in print: Dec 1, 2024
Discussion open until: Feb 28, 2025
ASCE Technical Topics:
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.