Gesture Recognition–Based Smart Training Assistant System for Construction Worker Earplug-Wearing Training
Publication: Journal of Construction Engineering and Management
Volume 146, Issue 12
Abstract
Thousands of construction workers suffer noise-induced hearing loss (NIHL) every year from excessive noise exposure on the job, which impairs the quality of their lives and increases the risk of injury. Properly wearing earplugs is very important onsite for worker hearing protection. However, the training provided in the current practice is minimal. Therefore, there is a need to develop an efficient and effective self-training method that can provide both accurate step-by-step earplug-wearing instructions and timely feedback through monitoring. With the development of artificial intelligence and wearable sensor technologies, the possibility of developing an advanced intelligent training method becomes plausible. Therefore, the objective of this paper is to develop a gesture recognition–based smart training assistant system that can automatically evaluate workers’ performance during their earplug-wearing self-training and provide timely feedback to rectify any mistakes. Through the system feasibility test and performance evaluation, the results show that the proposed system can achieve around 90% training accuracy and around 80% testing accuracy recognizing the classified forearm gestures of wearing earplugs for noise protection training using the developed artificial neural network (ANN) models for both hands. The proposed gesture recognition–based smart training assistant system will eventually help industries to improve the performance and safety of employees with low implementation costs.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Data generated or analyzed during the study are available from the corresponding author by request. Information about the Journal’s data-sharing policy can be found here: http://ascelibrary.org/doi/10.1061/(ASCE)CO.1943-7862.0001263.
Acknowledgments
We thank Weile Wei, Shengfeng Li, Yirui Deng, Deekshith Mandala, Shuai Yuan, and Tianyi Zhang for their pilot study in their course project.
References
Abraham, A. 2005. “Artificial neural networks.” In Handbook of measuring system design. Chichester, UK: Wiley.
Abreu, J. G., J. M. Teixeira, L. S. Figueiredo, and V. Teichrieb. 2016. “Evaluating sign language recognition using the Myo armband.” In Proc., 2016 XVIII Symp. on Virtual and Augmented Reality (SVR), 64–70. New York: IEEE.
Akhavian, R., and A. Behzadan. 2015. “Wearable sensor-based activity recognition for data-driven simulation of construction workers’ activities.” In Proc., 2015 Winter Simulation Conf. (WSC), 3333–3344. New York: IEEE.
Arief, Z., I. A. Sulistijono, and R. A. Ardiansyah. 2015. “Comparison of five time series EMG features extractions using Myo armband.” In Proc., 2015 Int. Electronics Symp. (IES), 11–14. New York: IEEE.
Babisch, W. 2002. “The noise/stress concept, risk assessment and research needs.” Noise Health 4 (16): 1–11.
Benalcázar, M. E., A. G. Jaramillo, A. Zea, A. Páez, and V. H. Andaluz. 2017. “Hand gesture recognition using machine learning and the Myo armband.” In Proc., 25th European on Signal Processing Conf. (EUSIPCO), 1040–1044. New York: IEEE.
Bitzer, S., and S. J. Kiebel. 2012. “Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks.” Biol. Cybern. 106 (4–5): 201–217. https://doi.org/10.1007/s00422-012-0490-x.
Boyali, A., N. Hashimoto, and O. Matsumoto. 2015. “Hand posture and gesture recognition using Myo armband and spectral collaborative representation based classification.” In Proc., 2015 IEEE 4th Global Conf. on Consumer Electronics (GCCE), 200–201. New York: IEEE.
Castillejo, P., J.-F. Martinez, J. Rodriguez-Molina, and A. Cuerva. 2013. “Integration of wearable devices in a wireless sensor network for an E-health application.” IEEE Wireless Commun. 20 (4): 38–49. https://doi.org/10.1109/MWC.2013.6590049.
Chau, T. 2017. “Machine learning for gesture recognition with electromyography.” Master thesis, Norwegian Univ. of Science and Technology. https://ntnuopen.ntnu.no/ntnu-xmlui/handle/11250/2459262.
Chen, J., C. R. Ahn, and S. Han. 2014. Detecting the hazards of lifting and carrying in construction through a coupled 3D sensing and IMUs sensing system. Comput. Civ. Build. Eng. 2014: 1110–1117. https://ascelibrary.org/doi/abs/10.1061/9780784413616.138.
Coyle, S., V. F. Curto, F. Benito-Lopez, L. Florea, and D. Diamond. 2014. “Wearable bio and chemical sensors.” In Wearable sensors, 65–83. Amsterdam, Netherlands: Elsevier.
Dayhoff, J. E., and J. M. DeLeo. 2001. “Artificial neural networks.” Cancer 91 (S8): 1615–1635. https://doi.org/10.1002/1097-0142(20010415)91:8+%3C1615::AID-CNCR1175%3E3.0.CO;2-L.
Ghasemzadeh, H., N. Amini, R. Saeedi, and M. Sarrafzadeh. 2015. “Power-aware computing in wearable sensor networks: An optimal feature selection.” IEEE Trans. Mob. Comput. 14 (4): 800–812. https://doi.org/10.1109/TMC.2014.2331969.
Gomaa, M. A. M., M. H. A. Elmagd, M. M. Elbadry, and R. M. A. Kader. 2014. “Depression, anxiety and stress scale in patients with tinnitus and hearing loss.” Eur. Arch. Oto-Rhino-Laryngology 271 (8): 2177–2184. https://doi.org/10.1007/s00405-013-2715-6.
Harman, B. I., H. Koseoglu, and C. O. Yigit. 2016. “Performance evaluation of IDW, kriging and multiquadric interpolation methods in producing noise mapping: A case study at the city of Isparta, Turkey.” App. Acoust. 112 (Nov): 147–157. https://doi.org/10.1016/j.apacoust.2016.05.024.
Hong, O. 2005. “Hearing loss among operating engineers in American construction industry.” Int. Arch. Occup. Environ. Health 78 (7): 565–574. https://doi.org/10.1007/s00420-005-0623-9.
Hong, O., D. L. Chin, L. A. Fiola, and A. S. Kazanis. 2013. “The effect of a booster intervention to promote hearing protection behavior in operating engineers.” Am. J. Ind. Med. 56 (2): 258–266. https://doi.org/10.1002/ajim.22091.
John, G., A. Grynevych, D. Welch, D. McBride, and P. Thorne. 2014. “Noise exposure of workers and the use of hearing protection equipment in New Zealand.” Arch. Environ. Occup. Health 69 (2): 69–80. https://doi.org/10.1080/19338244.2012.732122.
Joseph, A., J. Punch, M. Stephenson, N. Paneth, E. Wolfe, and W. Murphy. 2007. “The effects of training format on earplug performance.” Int. J. Audiology 46 (10): 609–618. https://doi.org/10.1080/14992020701438805.
Jutila, M., H. Rivas, P. Karhula, and S. Pantsar-Syväniemi. 2014. “Implementation of a wearable sensor vest for the safety and well-being of children.” Procedia Comput. Sci. 32: 888–893. https://doi.org/10.1016/j.procs.2014.05.507.
Kälsch, H., F. Hennig, S. Moebus, S. Möhlenkamp, N. Dragano, H. Jakobs, M. Memmesheimer, R. Erbel, K.-H. Jöckel, and B. Hoffmann. 2014. “Are air pollution and traffic noise independently associated with atherosclerosis: The Heinz Nixdorf recall study.” Eur. Heart J. 35 (13): 853–860. https://doi.org/10.1093/eurheartj/eht426.
Kefer, K., C. Holzmann, and R. D. Findling. 2017. “Evaluating the placement of arm-worn devices for recognizing variations of dynamic hand gestures.” J. Mobile Multimedia 12 (3–4): 225–242.
Kutafina, E., D. Laukamp, R. Bettermann, U. Schroeder, and S. M. Jonas. 2016. “Wearable sensors for elearning of manual tasks: Using forearm EMG in hand hygiene training.” Sensors 16 (8): 1221. https://doi.org/10.3390/s16081221.
LeCun, Y., Y. Bengio, and G. J. G. S. Hinton. 2015. “Deep learning.” Nature 521 (7553): 436. https://doi.org/10.1038/nature14539.
Leensen, M., J. Van Duivenbooden, and W. A. Dreschler. 2011. “A retrospective analysis of noise-induced hearing loss in the Dutch construction industry.” Int. Arch. Occup. Environ. Health 84 (5): 577–590. https://doi.org/10.1007/s00420-010-0606-3.
Lusk, S. L., O. S. Hong, D. L. Ronis, B. L. Eakin, M. J. Kerr, and M. R. Early. 1999. “Effectiveness of an intervention to increase construction workers’ use of hearing protection.” Human Factors: J. Human Factors Ergon. Soc. 41 (3): 487–494. https://doi.org/10.1518/001872099779610969.
Masterson, E. A., S. Tak, C. L. Themann, D. K. Wall, M. R. Groenewold, J. A. Deddens, and G. M. Calvert. 2013. “Prevalence of hearing loss in the United States by industry.” Am. J. Ind. Med. 56 (6): 670–681. https://doi.org/10.1002/ajim.22082.
Mendez, I., B. W. Hansen, C. M. Grabow, E. J. L. Smedegaard, N. B. Skogberg, X. J. Uth, A. Bruhn, B. Geng, and E. N. Kamavuako. 2017. “Evaluation of the Myo armband for the classification of hand motions.” In Proc., 2017 Int. Conf. on Rehabilitation Robotics (ICORR), 1211–1214. New York: IEEE.
Mohamad, T. H., M. Samadani, and C. Nataraj. 2018. “Rolling element bearing diagnostics using extended phase space topology.” J. Vib. Acoust. 140 (6): 061009. https://doi.org/10.1115/1.4040041.
Münzel, T., T. Gori, W. Babisch, and M. Basner. 2014. “Cardiovascular effects of environmental noise exposure.” Eur. Heart J. 35 (13): 829–836. https://doi.org/10.1093/eurheartj/ehu030.
Murphy, W. J., M. R. Stephenson, D. C. Byrne, B. Witt, and J. Duran. 2011. “Effects of training on hearing protector attenuation.” Noise Health 13 (51): 132. https://doi.org/10.4103/1463-1741.77215.
NIOSH (National Institute for Occupational Safety and Health). 2018. “Noise and hearing loss prevention.” Accessed July 7, 2020. https://www.cdc.gov/niosh/topics/noise/preventhearingloss/hearlosspreventprograms.html.
National Institutes of Health. 1990. “Noise and hearing loss. NIH consens statement 1990 Jan 22–24.” 8 (1): 1–24. https://consensus.nih.gov/1990/1990noisehearingloss076html.htm.
Neitzel, R., and N. Seixas. 2005. “The effectiveness of hearing protection among construction workers.” J. Occup. Environ. Hyg. 2 (4): 227–238. https://doi.org/10.1080/15459620590932154.
Ojha, V. K., A. Abraham, and V. Snášel. 2017. “Metaheuristic design of feed-forward neural networks: A review of two decades of research.” Eng. Appl. Artif. Intell. 60 (Apr): 97–116. https://doi.org/10.1016/j.engappai.2017.01.013.
OSHA (Occupational Safety and Health Administration). 2016. “Occupational noise exposure.” Accessed October 23, 2016. https://www.osha.gov/SLTC/noisehearingconservation/.
Safe Work Australia. 2010. Code of practice: Managing noise and preventing hearing loss at work (draft). Canberra, Australia: Safe Work Australia.
Schalkoff, R. J. 1997. Artificial neural networks. New York: McGraw-Hill.
Schulz, T. Y. 2008. “PPE management: What trainers need to know about fitting earplugs.” Accessed July 7, 2020. https://www.ehstoday.com/ppe/hearing-protection/article/21908402/ppe-management-what-trainers-need-to-know-about-fitting-earplugs.
Sliwinska-Kowalska, M., and A. Davis. 2012. “Noise-induced hearing loss.” Noise Health 14 (61): 274. https://doi.org/10.4103/1463-1741.104893.
Soh, P. J., G. A. Vandenbosch, M. Mercuri, and D. M. P. Schreurs. 2015. “Wearable wireless health monitoring: Current developments, challenges, and future trends.” IEEE Microwave Mag. 16 (4): 55–70. https://doi.org/10.1109/MMM.2015.2394021.
Suter, A. H. 2012. “Engineering controls for occupational noise exposure.” Sound Vib. 46: 24–31.
Tak, S., R. R. Davis, and G. M. Calvert. 2009. “Exposure to hazardous workplace noise and use of hearing protection devices among US workers—NHANES, 1999–2004.” Am. J. Ind. Med. 52 (5): 358–371. https://doi.org/10.1002/ajim.20690.
Teizer, J. 2015. “Wearable, wireless identification sensing platform: Self-monitoring alert and reporting technology for hazard avoidance and training (SmartHat).” J. Inf. Technol. Constr. 20 (19): 295–312.
Utley, W., and L. A. Miller. 1985. “Occupational noise exposure on construction sites.” App. Acoust. 18 (4): 293–303. https://doi.org/10.1016/0003-682X(85)90014-3.
Valero, E., A. Sivanathan, F. Bosché, and M. Abdel-Wahab. 2016. “Musculoskeletal disorders in construction: A review and a novel system for activity tracking with body area network.” Appl. Ergon. 54 (May): 120–130. https://doi.org/10.1016/j.apergo.2015.11.020.
Van Kempen, E., and W. Babisch. 2012. “The quantitative relationship between road traffic noise and hypertension: A meta-analysis.” J. Hypertens. 30 (6): 1075–1086. https://doi.org/10.1097/HJH.0b013e328352ac54.
Zhang, Z., K. Yang, J. Qian, and L. Zhang. 2019. “Real-time surface EMG pattern recognition for hand gestures based on an artificial neural network.” Sensors 19 (14): 3170. https://doi.org/10.3390/s19143170.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 146 • Issue 12 • December 2020
Copyright
© 2020 American Society of Civil Engineers.
History
Received: Mar 18, 2020
Accepted: Jul 7, 2020
Published online: Oct 13, 2020
Published in print: Dec 1, 2020
Discussion open until: Mar 13, 2021
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.