Human Intrusion Detection in Static Hazardous Areas at Construction Sites: Deep Learning–Based Method
Publication: Journal of Construction Engineering and Management
Volume 149, Issue 1
Abstract
Construction sites have complex environments and high accident rates. Human intrusion into static hazardous areas is a significant cause of accidents. Traditional engineering safety management mainly relies on manual methods, such as patrol inspection by safety supervisors, which is time-consuming and labor-intensive, and it is difficult to achieve a complete safety supervision. In recent years, the emergence of artificial intelligence technology and computer vision has provided a new scheme for intrusion detection. However, existing studies have used a single method for human intrusion judgment in static dangerous areas, without in-depth consideration of the influence of human posture, intrusion direction, and other factors. In this study, a computer vision–based intrusion detection method was developed, mainly aimed at static hazardous areas. The object detection was based on the You Only Look Once (YOLO) V5 module to extract the image feature information. Subsequently, the basic rule of intrusion judgment based on the key points of bounding boxes was formulated, in which the workers’ intrusion direction was recognized and postured using two auxiliary detection modules. Finally, the intrusion rule base was constructed as the basis for human intrusion detection, containing rules with different sensitivities for different intrusion states. The case study indicated that the precision and recall rate of the algorithm were 96.05% and 90.05%, respectively. Overall, this method can effectively address the defects of manual supervision in engineering safety management, reducing the probability of accident occurrence and enhancing safety at construction sites.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This research was supported by the Science Research Plan of Shanghai Municipal Science and Technology Committee (Grant No. 20dz1201301), and the 2021 Science Research Plan of Shanghai Housing and Urban-Rural Development Management Committee (Grant No. 2021-002-049).
References
Ahmad, M., I. Ahmed, and G. Jeon. 2021. “An IoT-enabled real-time overhead view person detection system based on Cascade-RCNN and transfer learning.” J. Real-Time Image Process. 18 (4): 1129–1139. https://doi.org/10.1007/s11554-021-01103-0.
Ahmed, I., M. Ahmad, A. Ahmad, and G. Jeon. 2021a. “Top view multiple people tracking by detection using deep SORT and YOLOv3 with transfer learning: Within 5G infrastructure.” Int. J. Mach. Learn. Cybern. 12 (11): 3053–3067. https://doi.org/10.1007/s13042-020-01220-5.
Ahmed, I., M. Anisetti, and G. Jeon. 2021b. “An IoT-based human detection system for complex industrial environment with deep learning architectures and transfer learning.” Int. J. Intell. Syst. https://doi.org/10.1002/int.22472.
Bochkovskiy, A., W. Chien-Yao, and H. Y. M. Liao. 2020. “YOLOv4: Optimal speed and accuracy of object detection arXiv.” Preprint, submitted April 23, 2020. https://arxiv.org/abs/2004.10934.
Boudjit, K., and N. Ramzan. 2021. “Human detection based on deep learning YOLO-v2 for real-time UAV applications.” J. Exp. Theor. Artif. Intell. 34 (3): 527–544. https://doi.org/10.1080/0952813X.2021.1907793.
Cai, Z. W., and N. Vasconcelos. 2018. “Cascade R-CNN: Delving into high quality object detection.” In Proc., 31st IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 6154–6162. New York: IEEE.
Chen, H., X. Luo, and Z. Zhu. 2021. “A worker posture coding scheme to link automatic and manual coding.” Autom. Constr. 125 (May): 103630. https://doi.org/10.1016/j.autcon.2021.103630.
Costin, A., N. Pradhananga, and J. Teizer. 2012. “Leveraging passive RFID technology for construction resource field mobility and status monitoring in a high-rise renovation project.” Autom. Constr. 24 (May): 1–15. https://doi.org/10.1016/j.autcon.2012.02.015.
Fang, Q., H. Li, X. Luo, L. Ding, H. Luo, and C. Li. 2018. “Computer vision aided inspection on falling prevention measures for steeplejacks in an aerial environment.” Autom. Constr. 93 (Sep): 148–164. https://doi.org/10.1016/j.autcon.2018.05.022.
Fang, W., B. Zhong, N. Zhao, P. E. Love, H. Luo, J. Xue, and S. Xu. 2019. “A deep learning-based approach for mitigating falls from height with computer vision: Convolutional neural network.” Adv. Eng. Inf. 39 (Jan): 170–177. https://doi.org/10.1016/j.aei.2018.12.005.
Guanhao, Y., F. Wei, J. Jintao, L. Qujiang, L. Xiuhao, G. Guangchao, and W. Weijun. 2020. “Face mask recognition system with YOLOV5 based on image recognition.” In Proc., 2020 IEEE 6th Int. Conf. on Computer and Communications, 1398–1404. New York: IEEE.
Guo, B. H. W., T. W. Yiu, and V. A. Gonzalez. 2016. “Predicting safety behavior in the construction industry: Development and test of an integrative model.” Saf. Sci. 84 (Apr): 1–11. https://doi.org/10.1016/j.ssci.2015.11.020.
Hamamura, T., T. Kimura, and H. Tsuji. 2021. “Detection of main persons in snapshots using deep neural networks.” In Proc., Int. Workshop on Advanced Imaging Technology (IWAIT). Kanagawa, Japan: Spie-Int Soc Optical Engineering.
Han, K. K., and M. Golparvar-Fard. 2015. “Appearance-based material classification for monitoring of operation-level construction progress using 4D BIM and site photologs.” Autom. Constr. 53 (May): 44–57. https://doi.org/10.1016/j.autcon.2015.02.007.
Han, S., S. Lee, and F. Peña-Mora. 2013. “Vision-based detection of unsafe actions of a construction worker: Case study of ladder climbing.” J. Comput. Civ. Eng. 27 (6): 635–644. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000279.
Han, S. U., S. H. Lee, and F. Peña-Mora. 2012. “A machine-learning classification approach to automatic detection of workers’ actions for behavior-based safety analysis.” In Proc., 2012 ASCE Int. Conf. on Computing in Civil Engineering, 65–72. Reston, VA: ASCE.
He, K. M., G. Gkioxari, P. Dollar, and R. Girshick. 2020. “Mask R-CNN.” IEEE Trans. Pattern Anal. Mach. Intell. 42 (2): 386–397. https://doi.org/10.1109/TPAMI.2018.2844175.
Hildreth, J., M. Vorster, and J. Martinez. 2005. “Reduction of short-interval GPS data for construction operations analysis.” J. Constr. Eng. Manage. 131 (8): 920–927. https://doi.org/10.1061/(ASCE)0733-9364(2005)131:8(920).
Hozhabri, M., M. Otterskog, N. Petrovic, and M. Ekstrom. 2016. Experimental comparison study of UWB technologies for static human detection. New York: IEEE Open Journal of the Microwave Theory and Techniques Society.
Hu, Q., Y. Bai, L. He, Q. Cai, S. Tang, G. Ma, J. Tan, and B. Liang. 2020. “Intelligent framework for worker-machine safety assessment.” J. Constr. Eng. Manage. 146 (5): 04020045. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001801.
Huang, J., et al. 2017. “Speed/accuracy trade-offs for modern convolutional object detectors.” In Proc., 30th IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR). New York: IEEE.
Jin, M., K. Jeong, S. Yoon, and D. S. Park. 2013. “Real-time pedestrian detection based on GMM and HOG cascade.” In Proc., 6th Int. Conf. on Machine Vision (ICMV). Bellingham, WA: Spie-Int Soc Optical Engineering.
Kim, H., H. S. Lee, M. Park, B. Chung, and S. Hwang. 2016. “Automated hazardous area identification using laborers’ actual and optimal routes.” Autom. Constr. 65 (May): 21–32. https://doi.org/10.1016/j.autcon.2016.01.006.
Kim, K., and M. Kim. 2012. “RFID-based location-sensing system for safety management.” Pers. Ubiquitous Comput. 16 (3): 235–243. https://doi.org/10.1007/s00779-011-0394-0.
Koch, C., K. Georgieva, V. Kasireddy, B. Akinci, and P. Fieguth. 2015. “A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure.” Adv. Eng. Inf. 29 (2): 196–210. https://doi.org/10.1016/j.aei.2015.01.008.
Li, X., S. Han, M. Gül, M. Al-Hussein, and M. El-Rich. 2018. “3D visualization-based ergonomic risk assessment and work modification framework and its validation for a lifting task.” J. Constr. Eng. Manage. 144 (1): 04017093. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001412.
Luo, X. C., H. Li, T. Huang, and T. Rose. 2016. “A field experiment of workers’ responses to proximity warnings of static safety hazards on construction sites.” Saf. Sci. 84 (Apr): 216–224. https://doi.org/10.1016/j.ssci.2015.12.026.
Maman, Z. S., M. A. A. Yazdi, L. A. Cavuoto, and F. M. Megahed. 2017. “A data-driven approach to modeling physical using wearable sensors.” Appl. Ergon. 65 (Nov): 515–529. https://doi.org/10.1016/j.apergo.2017.02.001.
Man-Woo, P., E. Palinginis, and I. Brilakis. 2012. “Detection of construction workers in video frames for automatic initialization of vision trackers.” In Proc., 2012 Construction Research Congress, 940–949. Reston, VA: ASCE.
Min, W. D., S. Zou, and J. Li. 2019. “Human fall detection using normalized shape aspect ratio.” Multimedia Tools Appl. 78 (11): 14331–14353. https://doi.org/10.1007/s11042-018-6794-7.
MOHURD (Ministry of Housing and Urban-Rural Development). 2020. “The MOHURD’s notigication on the safety accidents in housing and municipal works in 2019.” [In Chinese.] Stand. Eng. Constr. 2020 (7): 51–53.
Ning, X., and G. Guo. 2013. “Assessing spinal loading using the kinect depth sensor: A feasibility study.” IEEE Sens. J. 13 (4): 1139–1140. https://doi.org/10.1109/JSEN.2012.2230252.
njvisionpower. 2019. “Safety-helmet-wearing-dataset.” Accessed May 19, 2022. https://github.com/njvisionpower/Safety-Helmet-Wearing-Dataset.
OSHA (Occupational Safety and Health Administration). 2022a. “Commonly used statistics.” Accessed May 18, 2022. https://www.osha.gov/oshstats/commonstats.html.
OSHA (Occupational Safety and Health Administration). 2022b. “OSHA’s fall prevention campaign.” Accessed May 18, 2022. https://www.osha.gov/stop-falls.
Padilla, R., S. L. Netto, and E. A. B. da Silva. 2020. “A survey on performance metrics for object-detection algorithms.” In Proc., 27th Int. Conf. on Systems, Signals and Image Processing (IWSSIP), 237–242. New York: IEEE.
Pang, Y., Y. Yuan, X. Li, and J. Pan. 2011. “Efficient HOG human detection.” Signal Process. 91 (4): 773–781. https://doi.org/10.1016/j.sigpro.2010.08.010.
Poppe, R. 2010. “A survey on vision-based human action recognition.” Image Vision Comput. 28 (6): 976–990. https://doi.org/10.1016/j.imavis.2009.11.014.
Pradhananga, N., and J. Teizer. 2012. “Spatio-temporal safety analysis of construction site operations using GPS data.” In Proc., 2012 Construction Research Congress, 787–796. Reston, VA: ASCE.
Ray, S. J., and J. Teizer. 2012. “Real-time construction worker posture analysis for ergonomics training.” Adv. Eng. Inf. 26 (2): 439–455. https://doi.org/10.1016/j.aei.2012.02.011.
Redmon, J., and A. Farhadi. 2018. “YOLOv3: An incremental improvement.” Preprint, submitted April 8, 2018. https://arxiv.org/abs/1804.02767.
Ren, S. Q., K. M. He, R. Girshick, and J. Sun. 2017. “Faster R-CNN: Towards real-time object detection with region proposal networks.” IEEE Trans. Pattern Anal. Mach. Intell. 39 (6): 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031.
SAC (Standardization Administration of China). 1986. The classification for casualty accidents of enterprise staff and workers. National standard. Beijing: SAC.
SAC (Standardization Administration of China). 2008. Classification of work at heights. National standard. Beijing: SAC.
Seo, J., K. Yin, and S. Lee. 2016. “Automated postural ergonomic assessment using a computer vision-based posture classification.” In Proc., Construction Research Congress, 809–818. San Juan, PR: Univ. Puerto Rico Mayaguez.
Seo, J. O., and S. H. Lee. 2021. “Automated postural ergonomic risk assessment using vision-based posture classification.” Autom. Constr. 128 (Aug): 103725. https://doi.org/10.1016/j.autcon.2021.103725.
Ultralytics. 2022. “yolov5.” Accessed May 19, 2022. https://github.com/ultralytics/yolov5.
Wang, C.-L., J. Zhou, and S. Huang. 2012. “Motion human detection based on mixture of Gaussians and PCA-HOG.” Appl. Res. Comput. 29 (6): 2156–2160. https://doi.org/10.3969/j.issn.1001-3695.2012.06.041.
Wang, Z., Y. Wu, L. Yang, A. Thirunavukarasu, C. Evison, and Y. Zhao. 2021a. “Fast personal protective equipment detection for real construction sites using deep learning approaches.” Sensors 21 (10): 3478. https://doi.org/10.3390/s21103478.
Wang, Z., F. Xiao, N. Ye, R. Wang, and P. Yang. 2018. “A see-through-wall system for device-free human motion sensing based on battery-free RFID.” ACM Trans. Embedded Comput. Syst. 17 (1): 6. https://doi.org/10.1145/3055515.
Wang, Z. J., Y. M. Wu, L. C. Yang, A. Thirunavukarasu, C. Evison, and Y. F. Zhao. 2021b. “Fast personal protective equipment detection for real construction sites using deep learning approaches.” Sensors 21 (10): 3478. https://doi.org/10.3390/s21103478.
Wenping, L. 2015. “The schematic studies of construction accident warning system based on BIM and positioning technology.” [In Chinese.] Master’s dissertation, Dept. of Management Science and Engineering, Tsinghua Univ.
Xue, H., M. Liu, Y. Zhang, F. Liang, F. Qi, F. Chen, H. Lv, J. Wang, and Y. Zhang. 2017. “An algorithm based wavelet entropy for shadowing effect of human detection using ultra-wideband bio-radar.” Sensors 17 (10): 2255. https://doi.org/10.3390/s17102255.
Ye, L., Y. Kangning, L. Jie, W. Chunyu, and Y. Guangqiang. 2020. “A multi-task joint framework for real-time person search arXiv.” Preprint, submitted December 11, 2020. https://arxiv.org/abs/2012.06418.
Yosinski, J., J. Clune, Y. Bengio, and H. Lipson. 2014. “How transferable are features in deep neural networks?” In Proc., 28th Conf. on Neural Information Processing Systems (NIPS). La Jolla, CA: Neural Information Processing Systems.
Yu, R., Z. Zhang, and H. Guo. 2019a. “A data-driven management approach of construction workers’ unsafe behaviors.” [In Chinese.] China Saf. Sci. J. 29 (8): 130–135.
Yu, Y., X. Yang, H. Li, X. Luo, H. Guo, and Q. Fang. 2019b. “Joint-level vision-based ergonomic assessment tool for construction workers.” J. Constr. Eng. Manage. 145 (5): 04019025. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001647.
Zhong, K., and X. Z. Wang. 2019. “Wearing safety helmet detection in substation.” In Proc., 2nd IEEE Int. Conf. on Electronics and Communication Engineering (ICECE), 206–210. New York: IEEE.
Zhou, B., C. Wu, T. Wang, and D. Fang. 2018. “The influence of conscientiousness to safety behavior in construction projects.” [In Chinese.] China Civ. Eng. J. 51 (8): 89–95.
Zhou, X.-H., S.-L. Shen, Y.-S. Xu, and A.-N. Zhou. 2019. “Analysis of production safety in the construction industry of China in 2018.” Sustainability 11 (17): 4537. https://doi.org/10.3390/su11174537.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 149 • Issue 1 • January 2023
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Feb 16, 2022
Accepted: Jul 21, 2022
Published online: Oct 22, 2022
Published in print: Jan 1, 2023
Discussion open until: Mar 22, 2023
ASCE Technical Topics:
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Yejin Shin, Yujin Choi, Jaeseung Won, Taehoon Hong, Choongwan Koo, A New Benchmark Model for the Automated Detection and Classification of a Wide Range of Heavy Construction Equipment, Journal of Management in Engineering, 10.1061/JMENEA.MEENG-5630, 40, 2, (2024).
- Fan Zhang, Hongbo Liu, Zhihua Chen, Longxuan Wang, Qian Zhang, Liulu Guo, Collision Risk Warning Model for Construction Vehicles Based on YOLO and DeepSORT Algorithms, Journal of Construction Engineering and Management, 10.1061/JCEMD4.COENG-13860, 150, 6, (2024).
- He Huang, Hao Hu, Feng Xu, Zhipeng Zhang, Kinesiology-Inspired Assessment of Intrusion Risk Based on Human Motion Features, Journal of Construction Engineering and Management, 10.1061/JCEMD4.COENG-13638, 150, 7, (2024).
- Sepehr Khorshid, Siyuan Song, Kaiwen Chen, Peiyi Lyu, A Review of Non-Invasive Heat Stress Monitoring Systems for Construction Workers Using Wearable Device and Computer Vision Technology, Computing in Civil Engineering 2023, 10.1061/9780784485248.073, (606-613), (2024).
- Seunghyeon Wang, Mincheol Kim, Hyeonyong Hae, Mengqiu Cao, Juhyung Kim, The Development of a Rebar-Counting Model for Reinforced Concrete Columns: Using an Unmanned Aerial Vehicle and Deep-Learning Approach, Journal of Construction Engineering and Management, 10.1061/JCEMD4.COENG-13686, 149, 11, (2023).