Activity Recognition of Construction Equipment Using Generated Sound Data
Publication: Computing in Civil Engineering 2021
ABSTRACT
Automatically monitoring construction equipment and their activities using sound data has been an active research area in recent years. Despite their contribution to the literature, the previous studies can only be applied to isolated single-machine scenarios. However, most construction jobsites consist of multiple machines performing activities simultaneously, and the generated sound signals are mixed. Therefore, there is a need to develop a robust approach that identifies multiple machines’ activities. To do so, we propose a novel software-based method using a single-channel microphone and convolutional neural network (CNN). This method eliminates the need for using microphone arrays that require specific setup and configurations. We also propose a data augmentation method for generating synthetic mixed data to increase the method’s accuracy. We test the method on one synthetic and one real-world equipment sound mixture. The results demonstrate the capability of the proposed method in identifying activities with an accuracy higher than 80%.
Get full access to this chapter
View all available purchase options and get full access to this chapter.
REFERENCES
Cheng, C. F., Rashidi, A., Davenport, M. A., and Anderson, D. V. 2017. Activity analysis of construction equipment using audio signals and support vector machines. Automation in construction, 81, pp.240–253. DOI: https://doi.org/10.1016/j.autcon.2017.06.005.
Ham, Y., and Kamari, M. 2019. Automated content-based filtering for enhanced vision-based documentation in construction toward exploiting big visual data from drones. Automation in Construction, 105, p.102831. https://doi.org/10.1016/j.autcon.2019.102831.
Kamari, M., and Ham, Y. 2020, November. Analyzing Potential Risk of Wind-Induced Damage in Construction Sites and Neighboring Communities Using Large-Scale Visual Data from Drones. In Construction Research Congress 2020: Computer Applications, Tempe, Arizona, pp. 915–923. DOI: https://doi.org/10.1061/9780784482865.097.
Kamari, M., and Ham, Y. 2021. Vision-based volumetric measurements via deep learning-based point cloud segmentation for material management in jobsites. Automation in Construction, 121, p.103430. https://doi.org/10.1016/j.autcon.2020.103430.
Khosrowpour, A., Niebles, J. C., and Golparvar-Fard, M. 2014b. “Visionbased workface assessment using depth images for activity analysis of interior construction operations.” Automation in Construction 48 (Dec): pp.74–87. DOI: https://doi.org/10.1016/j.autcon.2014.08.003.
Kim, H., Ahn, C. R., Engelhaupt, D., and Lee, S. 2018a. “Application of dynamic time warping to the recognition of mixed equipment activities in cycle time measurement.” Automation in Construction 87 (Mar): pp.225–234. DOI: https://doi.org/10.1016/j.autcon.2017.12.014.
Kim, H., Ham, Y., Kim, W., Park, S., and Kim, H. 2019b. “Vision-based nonintrusive context documentation for earthmoving productivity simulation.” Automation in Construction 102 (Jun): pp.135–147. DOI: https://doi.org/10.1016/j.autcon.2019.02.006.
Liu, Z., and Li, S. 2020. A sound monitoring system for prevention of underground pipeline damage caused by construction. Automation in Construction, 113, pp.103125. https://doi.org/10.1016/j.autcon.2020.103125.
Luo, H., Wang, M., Wong, P. K. Y., and Cheng, J. C. 2020. Full body pose estimation of construction equipment using computer vision and deep learning techniques. Automation in construction, 110, pp.103016. https://doi.org/10.1016/j.autcon.2019.103016.
Rashid, K. M., and Louis, J. 2019. Times-series data augmentation and deep learning for construction equipment activity recognition. Advanced Engineering Informatics, 42, pp.100944. https://doi.org/10.3390/s19194286.
Rashid, K. M., and Louis, J. 2020. Activity identification in modular construction using audio signals and machine learning. Automation in construction, 119, pp.103361. https://doi.org/10.1016/j.autcon.2020.103361.
Ren, X., Zhu, Z., Germain, C., Belair, R., and Chen, Z. 2017. Automated monitoring of the utilization rate of onsite construction equipment. In Computing in Civil Engineering 2017, Seattle, Washington, pp. 74–81. DOI: https://doi.org/10.1061/9780784480830.010.
Sabillon, C., Rashidi, A., Samanta, B., Davenport, M. A., and Anderson, D. V. 2020. Audio-based bayesian model for productivity estimation of cyclic construction activities. Journal of Computing in Civil Engineering, 34(1), pp.04019048. DOI: https://doi.org/10.1061/(ASCE)CP.1943-5487.0000863.
Saruwatari, H., Kurita, S., Takeda, K., Itakura, F., Nishikawa, T., and Shikano, K. 2003. Blind source separation combining independent component analysis and beamforming. EURASIP Journal on Advances in Signal Processing, 2003(11), pp.569270. https://doi.org/10.1155/S1110865703305104.
Serviere, C., and Fabry, P. 2005. Principal component analysis and blind source separation of modulated sources for electro-mechanical systems diagnostic. Mechanical systems and signal processing, 19(6), pp.1293–1311. DOI: https://doi.org/10.1016/j.ymssp.2005.08.001.
Sherafat, B., Ahn, C. R., Akhavian, R., Behzadan, A. H., Golparvar-Fard, M., Kim, H., Lee, Y. C., Rashidi, A., and Azar, E. R. 2020. Automated Methods for Activity Recognition of Construction Workers and Equipment: State-of-the-Art Review. Journal of Construction Engineering and Management, 146(6), pp.03120002. DOI: https://doi.org/10.1061/(ASCE)CO.1943-7862.0001843.
Sherafat, B., Rashidi, A., and Song, S. 2020, November. A Software-Based Approach for Acoustical Modeling of Construction Job Sites with Multiple Operational Machines. In Construction Research Congress 2020: Computer Applications (pp. 886–895). Tempe, AZ, DOI: https://doi.org/10.1061/9780784482865.094.
Sherafat, B., Rashidi, A., Lee, Y. C., and Ahn, C. R. 2019. A hybrid kinematic-acoustic system for automated activity detection of construction equipment. Sensors, 19(19), pp.4286. https://doi.org/10.3390/s19194286.
Sherafat, B., Rashidi, A., Lee, Y. C., and Ahn, C. R. 2019. Automated activity recognition of construction equipment using a data fusion approach. In Computing in civil engineering 2019: Data, sensing, and analytics, Atlanta, Georgia, USA, pp. 1–8, DOI: https://doi.org/10.1061/9780784482438.001.
Torres Calderon, W., Roberts, D., and Golparvar-Fard, M. 2020. Synthesizing Pose Sequences from 3D Assets for Vision-Based Activity Analysis. Journal of Computing in Civil Engineering, 35(1), pp.04020052. DOI: https://doi.org/10.1061/(ASCE)CP.1943-5487.0000937.
Wang, J., Ye, K., Cao, J., Wang, T., Xue, A., Cheng, Y., and Yin, C. 2017. DOA estimation of excavation devices with ELM and MUSIC-based hybrid algorithm. Cognitive Computation, 9(4), pp.564–580. DOI: https://doi.org/10.1007/s12559-017-9475-3.
Wang, T., Cao, J., Wang, J., Lai, X., and Lin, Z. 2016, October. Excavation devices classification using enhanced acoustics by MVDR beamforming with a cross microphone array. In 2016 IEEE International Conference on Digital Signal Processing (DSP), Beijing, China, pp. 388–392. DOI: https://doi.org/10.1109/ICDSP.2016.7868585.
Zhang, T., Lee, Y. C., Scarpiniti, M., and Uncini, A. 2018, April. A supervised machine learning-based sound identification for construction activity monitoring and performance evaluation. In Construction Research Congress 2018, New Orleans, Louisiana, USA, pp. 358–366. DOI: https://doi.org/10.1061/9780784481264.035.
Information & Authors
Information
Published In
Computing in Civil Engineering 2021
Pages: 213 - 220
History
Published online: May 24, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.