Abstract
The ability to hear auditory safety cues of mobile equipment while wearing hearing protection equipment (HPE) is critical to preventing injuries and deaths in construction. Existing collision hazard detection models using proximity technologies have limited applicability due to the need for an expensive and complex deployment of sensing devices on every piece of construction equipment. This study proposes a more affordable collision prevention technology that uses audio signals to detect the presence of mobile equipment. The study addresses the problem by improving the auditory situational awareness for construction workers exposed to loud noises with a novel sound detection model that uses artificial intelligence (AI) to detect the sound of collision hazards buried in a great deal of ambient noises. This study included three phases: (1) collecting audio data of construction equipment, (2) developing a novel audio-based machine learning model for automated detection of collision hazards, and (3) conducting field experiments to investigate the system’s efficiency and latency. The outcomes showed that the proposed model detects equipment correctly and can timely notify the workers of hazardous situations.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This publication was made possible by CPWR–The Center for Construction Research and Training through cooperative Agreement No. U60-OH009762 from the National Institute of Occupational Safety and Health (NIOSH). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of CPWR or NIOSH.
References
Alamdari, N., F. Saki, A. Sehgal, and N. Kehtarnavaz. 2017. “An unsupervised noise classification smartphone app for hearing improvement devices.” In Proc., Signal Processing in Medicine and Biology Symp. (SPMB), 1–5. New York: IEEE.
Chae, S., and T. Yoshida. 2010. “Application of RFID technology to prevention of collision accident with heavy equipment.” Autom. Constr. 19 (3): 368–374. https://doi.org/10.1016/j.autcon.2009.12.008.
Cheng, C. F., A. Rashidi, M. A. Davenport, and D. Anderson. 2016. “Audio signal processing for activity recognition of construction heavy equipment.” In Proc., 2016 33rd Int. Symp. on Automation and Robotics in Construction, 642–650. Oulu, Finland: International Association for Automation and Robotics in Construction. https://doi.org/10.22260/ISARC2016/0078.
Cheng, C.-F., A. Rashidi, M. A. Davenport, and D. V. Anderson. 2017. “Activity analysis of construction equipment using audio signals and support vector machines.” Autom. Constr. 81 (4): 240–253. https://doi.org/10.1016/j.autcon.2017.06.005.
Cho, C., Y.-C. Lee, and T. Zhang. 2017. Sound recognition techniques for multi-layered construction activities and events. Reston, VA: ASCE. https://doi.org/10.1061/9780784480847.041.
Cowling, M., and R. Sitte. 2003. Comparison of techniques for environmental sound recognition. North Holland, Netherlands: Pattern Recognition Letters. https://doi.org/10.1016/S0167-8655(03)00147-8.
Dobie, R. A. 2005. “Estimating noise-induced permanent threshold shift from audiometric shape: The ISO-1999 model.” Ear Hear. 46 (6): 630–635. https://doi.org/10.1097/01.aud.0000188120.14321.76.
Durand, L. G., Z. Guo, H. N. Sabbah, and P. D. Stein. 1993. “Comparison of spectral techniques for computer-assisted classification of spectra of heart sounds in patients with porcine bioprosthetic valves.” Med. Biol. Eng. Comput. 31 (3): 229–236. https://doi.org/10.1007/bf02458041.
Eronen, A. J., V. T. Peltonen, J. T. Tuomi, A. P. Klapuri, S. Fagerlund, T. Sorsa, G. Lorho, and J. Huopaniemi. 2006. “Audio-based context recognition.” IEEE Trans. Audio Speech Lang. Process. 2006 (5): 321–329. https://doi.org/10.1109/TSA.2005.854103.
Gemmeke, J. F., D. P. W. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter. 2017. “Audio set: An ontology and human-labeled dataset for audio events.” In Proc., ICASSP, IEEE Int. Conf. on Acoustics, Speech and Signal Processing, 776–780. New York: IEEE.
Golovina, O., J. Teizer, and N. Pradhananga. 2016. “Heat map generation for predictive safety planning: Preventing struck-by and near miss interactions between workers-on-foot and construction equipment.” Autom. Constr. 71 (4): 99–115. https://doi.org/10.1016/j.autcon.2016.03.008.
Goodfellow, I., Y. Bengio, and A. Courville. 2016. Deep learning. Cambridge, MA: MIT Press.
Hasan, M. R., M. M. Hasan, and M. Z. Hossain. 2021. “How many Mel-frequency cepstral coefficients to be utilized in speech recognition? A study with the Bengali language.” J. Eng. 2021 (1): 21. https://doi.org/10.1049/tje2.12082.
Hinze, J. W., and J. Teizer. 2011. “Visibility-related fatalities related to construction equipment.” Saf. Sci. 49 (5): 709–718. https://doi.org/10.1016/j.ssci.2011.01.007.
Huang, Y., A. Hammad, and Z. Zhu. 2021. “Providing proximity alerts to workers on construction sites using Bluetooth Low Energy RTLS.” Autom. Constr. 132 (Dec): 103928. https://doi.org/10.3311/ccc2020-007.
Inoue, T., P. Vinayavekhin, S. Wang, D. Wood, A. Munawar, B. J. Ko, N. Greco, and R. Tachibana. 2019. “Shuffling and mixing data augmentation for environmental sound classification.” In Proc., Detection and Classification of Acoustic Scenes and Events 2019 Workshop. New York: New York Univ. https://doi.org/10.33682/wgyb-bt40.
Janjua, Z. H., M. Vecchio, M. Antonini, and F. Antonelli. 2019. “IRESE: An intelligent rare-event detection system using unsupervised learning on the IoT edge.” Eng. Appl. Artif. Intell. 84 (5): 41–50. https://doi.org/10.1016/j.engappai.2019.05.011.
Kim, D., M. Liu, S. Lee, and V. R. Kamat. 2019. “Remote proximity monitoring between mobile construction resources using camera-mounted UAVs.” Autom. Constr. 99 (5): 168–182. https://doi.org/10.1016/j.autcon.2018.12.014.
Kim, K., H. Kim, and H. Kim. 2017. “Image-based construction hazard avoidance system using augmented reality in wearable device.” Autom. Constr. 83 (45): 390–403. https://doi.org/10.1016/j.autcon.2018.12.014.
Lee, Y.-C., M. Scarpiniti, and A. Uncini. 2020a. “Advanced sound classifiers and performance analyses for accurate audio-based construction project monitoring.” J. Comput. Civ. Eng. 34 (5): 4020030. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000911.
Lee, Y.-C., M. Shariatfar, A. Rashidi, and H. W. Lee. 2020b. “Evidence-driven sound detection for prenotification and identification of construction safety hazards and accidents.” Autom. Constr. 113 (4): 103127. https://doi.org/10.1016/j.autcon.2020.103127.
Leonardo, M., N. Berardo, C. Alessandro, R. Luigi, and D. G. Giuseppe Martino. 2020. “Development of a digital twin model for real-time assessment of collision hazards.” In Proc., Creative Construction e-Conf. 2020. Budapest, Hungary: Budapest Univ. of Technology and Economics. https://doi.org/10.3311/CCC2020-003.
Liu, Z., and S. Li. 2020. “A sound monitoring system for prevention of underground pipeline damage caused by construction.” Autom. Constr. 113 (May): 103125. https://doi.org/10.1016/j.autcon.2020.103125.
Ma, L., B. Milner, and D. Smith. 2006. “Acoustic environment classification.” In ACM transactions on speech and language processing. New York: Association for Computing Machinery.
Marks, E. D., and J. Teizer. 2013. “Method for testing proximity detection and alert technology for safe construction equipment operation.” Construct. Manage. Econ. 31 (6): 636–646. https://doi.org/10.1080/01446193.2013.783705.
Marrone, S., G. Piantadosi, S. Olivieri, and C. Sansone. 2019. “Reproducibility of deep CNN for biomedical image processing across frameworks and architectures.” In Proc., European Signal Processing Conf. Piscataway, NJ: IEEE. https://doi.org/10.23919/EUSIPCO.2019.8902690.
Massoudi, M., S. Verma, and R. Jain. 2021. “Urban sound classification using CNN.” In Proc., 6th Int. Conf. on Inventive Computation Technologies. Piscataway, NJ: IEEE. https://doi.org/10.1109/ICICT50816.2021.9358621.
Papakipos, Z., and J. Bitton. 2022. “AugLy: Data augmentations for adversarial robustness.” In Proc., IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 156–163. Piscataway, NJ: IEEE. https://doi.org/10.48550/arXiv.2201.06494.
Park, J., X. Yang, Y. K. Cho, and J. Seo. 2017. “Improving dynamic proximity sensing and processing for smart work-zone safety.” Autom. Constr. 84 (1): 111–120. https://doi.org/10.1016/j.autcon.2017.08.025.
Piczak, K. J. 2015. “Environmental sound classification with convolutional neural networks.” In Proc., 2015 IEEE 25th Int. Workshop on Machine Learning for Signal Processing (MLSP), 1–6. Piscataway, NJ: IEEE. https://doi.org/10.1109/MLSP.2015.7324337.
Pradhananga, N., and J. Teizer. 2013. “Automatic spatio-temporal analysis of construction site equipment operations using GPS data.” Autom. Constr. 29 (4): 107–122. https://doi.org/10.1016/j.autcon.2012.09.004.
Sabillon, C. A., A. Rashidi, B. Samanta, C. F. Cheng, M. A. Davenport, and D. V. Anderson. 2018. “A productivity forecasting system for construction cyclic operations using audio signals and a Bayesian approach.” In Proc., Construction Research Congress 2018: Construction Information Technology–Selected Papers from the Construction Research Congress 2018, 295–304. Reston, VA: ASCE.
Sakhakarmi, S., J. W. Park, and A. Singh. 2021. “Tactile-based wearable system for improved hazard perception of worker and equipment collision.” Autom. Constr. 125 (May): 103613. https://doi.org/10.1016/j.autcon.2021.103613.
Salamon, J., and J. P. Bello. 2015. “Unsupervised feature learning for urban sound classification.” In Proc., ICASSP, IEEE Int. Conf. on Acoustics, Speech and Signal Processing, 171–175. New York: IEEE.
Salamon, J., C. Jacoby, and J. P. Bello. 2014. “A dataset and taxonomy for urban sound research.” In Proc., 2014 ACM Conf. on Multimedia, 1041–1044. New York: IEEE.
Salekin, A., S. Ghaffarzadegan, Z. Feng, and J. Stankovic. 2019. A real-time audio monitoring framework with limited data for constrained devices, 98–105. New York: IEEE.
Sherafat, R., A. Rashidi, Y.-C. Lee, and L. Ahn. 2019. “A hybrid kinematic-acoustic system for automated activity detection of construction equipment.” Sensors 19 (19): 4286. https://doi.org/10.3390/s19194286.
Son, H., H. Seong, H. Choi, and C. Kim. 2019. “Real-time vision-based warning system for prevention of collisions between workers and heavy equipment.” J. Comput. Civ. Eng. 33 (5): 4019029. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000845.
Srivastava, N., G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. 2014. “Dropout: A simple way to prevent neural networks from overfitting.” J. Mach. Learn. Res. 15 (1): 1929–1958. https://doi.org/10.5555/2627435.2670313.
Tarzia, S. P., P. A. Dinda, R. P. Dick, and G. Memik. 2011. “Indoor localization without infrastructure using the acoustic background spectrum.” In Proc., 9th Int. Conf. on Mobile Systems, Applications and Services and Co-located Workshops. New York: IEEE. https://doi.org/10.1145/1999995.2000011.
Tsakanikas, V., and T. Dagiuklas. 2018. “Video surveillance systems-current status and future trends.” Comput. Electr. Eng. 70 (4): 18–21. https://doi.org/10.1016/j.compeleceng.2017.11.011.
Vinnik, E., P. M. Itskov, and E. Balaban. 2011. “Individual differences in sound-in-noise perception are related to the strength of short-latency neural responses to noise.” PLoS One 6 (2): e17266. https://doi.org/10.1371/journal.pone.0017266.
Wang, J., and S. Razavi. 2016a. “Two 4D models effective in reducing false alarms for struck-by-equipment hazard prevention.” J. Comput. Civ. Eng. 30 (6): 4016031. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000589.
Wang, J., and S. Razavi. 2018. “Spatiotemporal network-based model for dynamic risk analysis on struck-by-equipment hazard.” J. Comput. Civ. Eng. 32 (2): 4017089. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000732.
Wang, J., and S. N. Razavi. 2016b. “Low false alarm rate model for unsafe-proximity detection in construction.” J. Comput. Civ. Eng. 30 (2): 4015005. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000470.
Wei, W., C. Wang, and Y. Lee. 2017. “BIM-Based construction noise hazard prediction and visualization for occupational safety and health awareness improvement.” Comput. Civ. Eng. 2017 (1): 262–269. https://doi.org/10.1061/9780784480823.032.
Wieczorkowska, A., E. Kubera, T. Słowik, and K. Skrzypiec. 2018. “Spectral features for audio based vehicle and engine classification.” J. Intell. Inf. Syst. 50 (2): 265–290. https://doi.org/10.1007/s10844-017-0459-2.
Xiang, J., M. F. McKinney, K. Fitz, and T. Zhang. 2010. “Evaluation of sound classification algorithms for hearing aid applications.” In Proc., 2010 IEEE Int. Conf. on Acoustics, Speech and Signal Processing, 185–188. New York: IEEE.
Xie, J., K. Hu, M. Zhu, J. Yu, and Q. Zhu. 2019a. “Investigation of different CNN-Based models for improved bird sound classification.” IEEE Access 2019 (1): 7. https://doi.org/10.1109/ACCESS.2019.2957572.
Xie, Y., Y.-C. Lee, M. Shariatfar, Z. D. Zhang, A. Rashidi, and H. W. Lee. 2019b. “Historical accident and injury database-driven audio-based autonomous construction safety surveillance.” In Computing in civil engineering 2019: Data, sensing, and analytics, 105–113. Reston, VA: ASCE.
Zhang, T., Y. C. Lee, M. Scarpiniti, and A. Uncini. 2018. “A supervised machine learning-based sound identification for construction activity monitoring and performance evaluation.” In Proc., Construction Research Congress 2018: Construction Information Technology—Selected Papers from the Construction Research Congress 2018, 358–366. Reston, VA: ASCE.
Zhang, X., Y. Zou, and W. Shi. 2017. “Dilated convolution neural network with LeakyReLU for environmental sound classification.” In Proc., Int. Conf. on Digital Signal Processing. Piscataway, NJ: IEEE. https://doi.org/10.1109/ICDSP.2017.8096153.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 149 • Issue 1 • January 2023
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Apr 11, 2022
Accepted: Sep 9, 2022
Published online: Nov 12, 2022
Published in print: Jan 1, 2023
Discussion open until: Apr 12, 2023
ASCE Technical Topics:
- Artificial intelligence and machine learning
- Business management
- Computer programming
- Computing in civil engineering
- Construction equipment
- Disaster risk management
- Employment
- Engineering fundamentals
- Equipment and machinery
- Field tests
- Infrastructure
- Labor
- Occupational safety
- Personnel management
- Practice and Profession
- Public administration
- Public health and safety
- Risk management
- Safety
- Tests (by type)
- Traffic engineering
- Traffic management
- Traffic safety
- Transportation engineering
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Kehinde Elelu, Tuyen Le, Chau Le, Augmented Hearing of Auditory Safety Cues for Construction Workers: A Systematic Literature Review, Sensors, 10.3390/s22239135, 22, 23, (9135), (2022).