Improved Vision-Based Method for Detection of Unauthorized Intrusion by Construction Sites Workers
Publication: Journal of Construction Engineering and Management
Volume 149, Issue 7
Abstract
The construction site environment is quite complex with many dangerous hazards (e.g., foundation pits, holes). To avoid injuries, workers must wear helmets that are color-coded for the specific type of work, which is helpful to identify whether workers are in permitted areas. Therefore, it is possible to identify unauthorized intrusion by classifying the safety helmets. This study proposes a vision-based method called Helmet–Yolov5 to automatically detect unauthorized intrusions by workers on construction sites. Multiple improvement measures are made to enhance the model performance. First, the attention mechanism is used to enhance the weights of object regions in the image, which makes the detection of small objects more effective. Second, atrous spatial pyramid pooling is adopted to preserve the detail information of the image. Third, the universal upsampling operator is introduced to fuse image features at different scales. To verify the effectiveness of the improved model, images collected from a real construction site are used to build a large-scale image dataset of safety helmets for model testing. It shows that the proposed Helmet-Yolov5 model is more accurate than the original Yolov5 model, also with high inference speed. Compared to other state-of-the-art models (e.g., Yolov4), the Helmet-Yolov5 model has considerable advantages in term of high detection accuracy and efficiency.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This work was supported by the Zhejiang Provincial Natural Science Foundation (LR23E080003) and the Zhejiang Provincial Key Research and Development Program (2021C03154).
References
BSI (British Standard Institute). 2012. Industrial safety helmets. BS EN 397. London: BSI.
Chen, L. C., G. Papandreou, F. Schroff, and H. Adam. 2017. “Rethinking atrous convolution for semantic image segmentation.” Preprint, submitted June 17, 2017. http://arxiv.org/abs/1706.05587.
Chen, S., and K. Demachi. 2021. “Towards on-site hazards identification of improper use of personal protective equipment using deep learning-based geometric relationships and hierarchical scene graph.” Autom. Constr. 125 (May): 103619. https://doi.org/10.1016/j.autcon.2021.103619.
Deng, J., W. Dong, R. Socher, L. J. Li, K. Li, and F. F. Li. 2009. “ImageNet: A large-scale hierarchical image database.” In Proc., 2009 IEEE Conf. on Computer Vision and Pattern Recognition, 248–255. New York: IEEE.
Ding, C. B., J. X. Huang, and D. P. Fang. 2004. “The accident causation and treatment in construction Industry of China.” China Civ. Eng. J. 37 (8): 77–82.
Dipu, N. M., S. A. Shohan, and K. M. Salam. 2021. “Deep learning-based brain tumor detection and classification.” In Proc., 2021 Int. Conf. on Intelligent Technologies (CONIT), 1–6. New York: IEEE.
Fang, Q., H. Li, X. C. Luo, L. Y. Ding, H. B. Luo, and C. Q. Li. 2018a. “Computer vision aided inspection on falling prevention measures for steeplejacks in an aerial environment.” Autom. Constr. 93 (Sep): 148–164. https://doi.org/10.1016/j.autcon.2018.05.022.
Fang, Q., H. Li, X. C. Luo, L. Y. Ding, H. B. Luo, T. M. Rose, and W. P. An. 2018b. “Detecting non-hardhat-use by a deep learning method from far-field surveillance videos.” Autom. Constr. 85 (Jan): 1–9. https://doi.org/10.1016/j.autcon.2017.09.018.
Fang, Q., H. Li, X. C. Luo, L. Y. Ding, H. B. Luo, T. M. Rose, W. P. An, and Y. T. Yu. 2018c. “A deep learning-based method for detecting non-certified work on construction sites.” Autom. Constr. 35 (Jan): 56–68. https://doi.org/10.1016/j.aei.2018.01.001.
Girshick, R. 2015. “Fast R-CNN.” In Proc., 2015 IEEE Int. Conf. on Computer Vision (ICCV), 1440–1448. New York: IEEE.
Girshick, R., J. Donahue, T. Darrell, and J. Malik. 2014. “Rich feature hierarchies for accurate object detection and semantic segmentation.” In Proc., 2014 IEEE Conf. on Computer Vision and Pattern Recognition. New York: IEEE.
Han, G., M. Zhu, X. Zhao, and H. Gao. 2021. “Method based on the cross-layer attention mechanism and multiscale perception for safety helmet-wearing detection.” Comput. Electr. Eng. 95 (Oct): 107458. https://doi.org/10.1016/j.compeleceng.2021.107458.
He, K. M., X. Y. Zhang, S. Q. Ren, and J. Sun. 2015. “Spatial pyramid pooling in deep convolutional networks for visual recognition.” IEEE Trans. Pattern Anal. Mach. Intell. 37 (9): 1904–1916. https://doi.org/10.1109/TPAMI.2015.2389824.
Huang, L., Q. Fu, M. He, D. Jiang, and Z. Hao. 2021. “Detection algorithm of safety helmet wearing based on deep learning.” Concurrency Comput. Pract. Exper. 33 (13): e6234. https://doi.org/10.1002/cpe.6234.
Jin, R., H. Zhang, D. H. Liu, and X. Z. Yan. 2020. “IoT-based detecting, locating and alarming of unauthorized intrusion on construction sites.” Autom. Constr. 118 (Oct): 103278. https://doi.org/10.1016/j.autcon.2020.103278.
Jocher, G., et al. 2021. “ultralytics/yolov5: v5.0 (version 5.0).” Accessed April 12, 2021. https://github.com/ultralytics/yolov5/tree/v5.0.
Lee, U. K., J. H. Kim, H. Cho, and K. I. Kang. 2009. “Development of a mobile safety monitoring system for construction sites.” Autom. Constr. 18 (3): 258–264. https://doi.org/10.1016/j.autcon.2008.08.002.
Li, H., G. Chan, T. Huang, M. Skitmore, Y. E. Terry, E. Luo, J. Chung, X. S. Chan, and Y. F. Li. 2015. “Chirp-spread-spectrum-based real time location system for construction safety management: A case study.” Autom. Constr. 55 (Jul): 58–65. https://doi.org/10.1016/j.autcon.2015.03.024.
Lin, T. Y., M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick. 2014. “Microsoft COCO: Common objects in context.” In Proc., Computer Vision—ECCV 2014, 8693. Berlin: Springer.
Liu, W., D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, and A. C. Berg. 2016. “SSD: Single shot multibox detector.” In Proc., European Conf. on Computer Vision, 21–37. Berlin: Springer.
Liu, W. P. 2015. The schematic studies of construction accident warning system based on BIM and positioning technology. Beijing: Tsinghua Univ.
Ministry of Housing and Urban-Rural Development of the People’s Republic of China. 2020. “Circular of the general office of the ministry of housing and urban-rural development on the production safety accidents of housing and municipal engineering in 2019.” Accessed June 19, 2020. https://www.mohurd.gov.cn/gongkai/zhengce/zhengcefilelib/202006/20200624_246031.html.
OSHA (Occupational Safety and Health Administration). 2015. “State OSHA annual report.” Accessed December 1, 2015. https://www.dir.ca.gov/dosh/reports/State-OSHA-Annual-Report-(SOAR)-FY2015.pdf.
Ott, P., and M. Everingham. 2009. “Implicit color segmentation features for pedestrian and object detection.” In Vol. 2009 of Proc., 2009 IEEE 12th Int. Conf. on Computer Vision, 723–730. New York: IEEE.
Park, M. W., N. Elsafty, and Z. H. Zhu. 2015. “Hardhat-wearing detection for enhancing on-site safety of construction workers.” J. Constr. Eng. Manage. 141 (9): 04015024. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000974.
Redmon, J., S. Divvala, R. Girshick, and A. Farhadi. 2016. “You only look once: Unified, real-time object detection.” In Proc., 2016 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 779–788. New York: IEEE.
Redmon, J., and A. Farhadi. 2017. “YOLO9000: Better, faster, stronger.” In Proc., 2017 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 6517–6525. New York: IEEE.
Redmon, J., and A. Farhadi. 2018. “YOLOv3: An incremental improvement.” Preprint, submitted April 8, 2018. http://arxiv.org/abs/1804.02767.
Ren, S. Q., K. M. He, R. Girshick, and J. Sun. 2017. “Faster R-CNN: Towards real-time object detection with region proposal networks.” IEEE Trans. Pattern Anal. Mach. Intell. 39 (6): 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031.
Seokho, C., H. C. Carlos, and D. Y. Kim. 2009. “A methodology for object identification and tracking in construction based on spatial modeling and image matching techniques.” Comput.-Aided Civ. Infrastruct. Eng. 24 (3): 199–211. https://doi.org/10.1111/j.1467-8667.2008.00580.x.
Shrestha, K., P. P. Shrestha, D. Bajracharya, and E. A. Yfantis. 2015. “Hard-hat detection for construction safety visualization.” J. Constr. Eng. 2015 (1): 1–8.
Song, X. Y., and W. Gu. 2021. “Multi-objective real-time vehicle detection method based on yolov5.” In Proc., 2021 Int. Symp. on Artificial Intelligence and Its Application on Media (ISAIAM), 142–145. New York: IEEE.
Sukkar, M., D. Kumar, and J. Sindha. 2021. “Real-time pedestrians detection by YOLOv5.” In Proc., 12th Int. Conf. on Computing Communication and Networking Technologies, 1–6. New York: IEEE.
Umeokafor, N., K. Evaggelinos, S. Lundy, D. Isaac, S. Allan, O. Igwegbe, K. Umeokafor, and B. Umeadi. 2014. “The pattern of occupational accidents, injuries, accident causal factors and intervention in Nigerian factories.” Dev. Country Stud. 4 (15): 119–127.
Wang, C. Y., H. Y. M. Liao, Y. H. Wu, P. Y. Chen, J. W. Hsieh, and I. H. Yeh. 2020a. “CSPNet: A new backbone that can enhance learning capability of CNN.” In Proc., 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), 1571–1580. New York: IEEE.
Wang, J. Q., K. Chen, R. Xu, Z. W. Liu, C. C. Loy, and D. H. Lin. 2019. “CARAFE: Content-aware reassembly of features.” In Proc., 2019 IEEE/CVF Int. Conf. on Computer Vision (ICCV), 3007–3016. New York: IEEE.
Wang, W., S. K. Lyu, Y. G. Zhang, C. N. Zhao, and H. G. He. 2020b. “Research on early warning of intrusion into hazardous construction area based on BIM and machine vision technology.” Saf. Environ. Eng. 27 (2): 196–203. https://doi.org/10.13578/j.cnki.issn.1671-1556.2020.02.027.
Waranusast, R., N. Bundon, V. Timtong, C. Tangnoi, and P. Pattanathaburt. 2013. “Machine vision techniques for motorcycle safety helmet detection.” In Proc., 28th Int. Conf. on Image and Vision Computing New Zealand (IVCNZ 2013), 35–40. New York: IEEE.
Woo, S., J. Park, J. Y. Lee, and I. S. Kweon. 2018. “CBAM: Convolutional block attention module.” In Proc., Computer Vision—ECCV 2018, 11211. Berlin: Springer.
Wu, J., N. Cai, W. Chen, H. Wang, and G. Wang. 2019. “Automatic detection of hardhats worn by construction personnel: A deep learning approach and benchmark dataset.” Autom. Constr. 106 (Oct): 102894. https://doi.org/10.1016/j.autcon.2019.102894.
Yan, X. Z., H. Zhang, and H. Li. 2020. “Computer vision-based recognition of 3D relationship between construction entities for monitoring struck-by accidents.” Comput.-Aided Civ. Infrastruct. Eng. 35 (9): 1023–1038. https://doi.org/10.1111/mice.12536.
Zhang, H., X. Z. Yan, H. Li, R. Jin, and H. F. Fu. 2019. “Real-time alarming, monitoring, and locating for non-hard-hat use in construction.” J. Constr. Eng. Manage. 145 (3): 04019006. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001629.
Zheng, Z., P. Wang, W. Liu, J. Li, R. Ye, and D. Ren. 2020. “Distance-IoU loss: Faster and better learning for bounding box regression.” In Vol. 34 of Proc., AAAI Conf. on Artificial Intelligence, 12993–13000. Palo Alto, CA: Association for the Advancement of Artificial Intelligence.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 149 • Issue 7 • July 2023
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Oct 28, 2022
Accepted: Feb 3, 2023
Published online: Apr 19, 2023
Published in print: Jul 1, 2023
Discussion open until: Sep 19, 2023
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.