A Multiscale Fusion YOLOV3-Based Model for Human Abnormal Behavior Detection in Special Scenarios
Publication: Journal of Transportation Engineering, Part A: Systems
Volume 149, Issue 2
Abstract
Urban public security incidents are prone to occur. Better understanding of pedestrian abnormal behavior and trajectory in crowded places is conducive to crowd management and safety monitoring. A novel pedestrian abnormal behavior detection model (PABDM) is proposed to identify crowd behavior under abnormal scenarios. This model originated from a multiscale fusion you only look once (YOLO) version 3 (V3) algorithm and was trained using the PASCAL visual object classes (VOC) in combination with an abnormal pedestrian data set (APD), denoted as VOC+APD. Compared with YOLOV3-VOC, single-stage detectors (SSD)-VOC, and , the proposed model has notable advantages in prediction accuracy and detection efficiency. The results show that the network loss function of the model tends to be stable after 500 epochs, and its detection accuracy is 6% higher than the average accuracy of the compared models. This proposed model also effectively solves the problem of missing detection caused by edge target, fuzzy target, and small target in abnormal state human detection. The research results are of great significance for real-time crowd monitoring in complex scenes.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
The data used to support the findings of this study are available from the corresponding author upon request.
Acknowledgments
This paper was supported by Beijing Social Science Fund Project No. 2020GLB020.
References
Adarsh, P., P. Rathi, and M. Kumar. 2020. “YOLO v3-Tiny: Object detection and recognition using one stage improved model.” In Proc., 6th Int. Conf. on Advanced Computing and Communication Systems (ICACCS). New York: IEEE.
Albawi, S., T. A. Mohammed, and S. Al-Zawi. 2017. “Understanding of a convolutional neural network.” In Proc., Int. Conf. on Engineering and Technology (ICET). New York: IEEE.
Ammar, A., A. Koubaa, M. Ahmed, and A. Saad. 2019. “Aerial images processing for car detection using convolutional neural networks: Comparison between Faster R-CNN and YOLOv3.” Preprint, submitted October 16, 2019. http://arxiv.org/abs/1910.07234.
Benjdira, B., T. Khursheed, A. Koubaa, A. Ammar, and K. Ouni. 2019. “Car detection using unmanned aerial vehicles: Comparison between Faster R-CNN and YOLOv3.” In Proc., 1st Int. Conf. on Unmanned Vehicle Systems-Oman (UVS). New York: IEEE.
Bernardi, M. L., M. Cimitile, F. Martinelli, and F. Mercaldo. 2018. “Driver and path detection through time-series classification.” J. Adv. Transp. 2018 (Mar): 1758731. https://doi.org/10.1155/2018/1758731.
Dalal, N., and B. Triggs. 2005. “Histograms of oriented gradients for human detection.” In Proc., IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR’05). New York: IEEE.
Do, T., M. Haghani, and M. Sarvi. 2016. “Group and single pedestrian behavior in crowd dynamics.” Transp. Res. Rec. 2540 (1): 13–19. https://doi.org/10.3141/2540-02.
Dollár, P., R. Appel, S. Belongie, and P. Perona. 2014. “Fast feature pyramids for object detection.” IEEE Trans. Pattern Anal. Mach. Intell. 36 (8): 1532–1545. https://doi.org/10.1109/TPAMI.2014.2300479.
Gayathri, H., P. M. Aparna, and A. Verma. 2017. “A review of studies on understanding crowd dynamics in the context of crowd safety in mass religious gatherings.” Int. J. Disaster Risk Reduct. 25 (Oct): 82–91. https://doi.org/10.1016/j.ijdrr.2017.07.017.
Girshick, R., J. Donahue, T. Darrell, and J. Malik. 2014. “Rich feature hierarchies for accurate object detection and semantic segmentation.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition. New York: IEEE.
Girshick, R. B. 2015. “Fast R-CNN.” In Proc., IEEE Int. Conf. on Computer Vision (ICCV), 1440–1448. New York: IEEE.
Haghani, M., and M. Sarvi. 2018. “Crowd behavior and motion: Empirical methods.” Transp. Res. Part B Methodol. 107 (Jan): 253–294. https://doi.org/10.1016/j.trb.2017.06.017.
Haghani, M., M. Sarvi, and Z. Shahhoseini. 2020. “Evacuation behavior of crowds under high and low levels of urgency: Experiments of reaction time, exit choice and exit-choice adaptation.” Saf. Sci. 126 (Jun): 104679. https://doi.org/10.1016/j.ssci.2020.104679.
Henderson, P., and V. Ferrari. 2017. “End-to-end training of object class detectors for mean average precision.” In Proc., Computer Vision—ACCV 2016. Cham, Switzerland: Springer International Publishing.
Henriques, J. F., and A. Vedaldi. 2017. “Warped convolutions: Efficient invariance to spatial transformations.” In Vol. 70 of Proc., 34th Int. Conf. on Machine Learning, 1461–1469. New York: Journal of Machine Learning Research.
Kang, K., et al. 2018. “T-CNN: Tubelets with convolutional neural networks for object detection from videos.” IEEE Trans. Circuits Syst. Video Technol. 28 (10): 2896–2907. https://doi.org/10.1109/TCSVT.2017.2736553.
Kim, S. H., S. C. Lim, and D. Y. Kim. 2018. “Intelligent intrusion detection system featuring a virtual fence, active intruder detection, classification, tracking, and action recognition.” Ann. Nucl. Energy 112 (Feb): 845–855. https://doi.org/10.1016/j.anucene.2017.11.026.
Kingma, D. P., and J. Ba. 2015. “Adam: A method for stochastic optimization.” Preprint, submitted December 22, 2014. http://arxiv.org/abs/1412.6980.
Kok, V. J., M. K. Lim, and C. S. Chan. 2016. “Crowd behavior analysis: A review where physics meets biology.” Neurocomputing 177 (Feb): 342–362. https://doi.org/10.1016/j.neucom.2015.11.021.
Lee, J., J. Won, and J. Lee. 2018. “Crowd simulation by deep reinforcement learning.” In Proc., 11th Annual Int. Conf. on Motion, Interaction, and Games (MIG ’18), 1–7. New York: Association for Computing Machinery. https://doi.org/10.1145/3274247.3274510.
Li, Z., and W. Xu. 2020. “Pedestrian evacuation within limited-space buildings based on different exit design schemes.” Saf. Sci. 124 (Apr): 104575. https://doi.org/10.1016/j.ssci.2019.104575.
Lin, L., X. Wang, W. Yang, and J.-H. Lai. 2014. “Discriminatively trained and-or graph models for object shape detection.” IEEE Trans. Pattern Anal. Mach. Intell. 37 (5): 959–972. https://doi.org/10.1109/TPAMI.2014.2359888.
Lin, T. Y., P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie. 2017. “Feature pyramid networks for object detection.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). New York: IEEE.
Lovreglio, R., M. Spearpoint, and M. Girault. 2019. “The impact of sampling methods on evacuation model convergence and egress time.” Reliab. Eng. Syst. Saf. 185 (May): 24–34. https://doi.org/10.1016/j.ress.2018.12.015.
Lowe, D. G. 2004. “Distinctive image features from scale-invariant keypoints.” Int. J. Comput. Vision 60 (2): 91–110. https://doi.org/10.1023/B:VISI.0000029664.99615.94.
Melnikov, V., V. V. Krzhizhanovskaya, and P. M. A. Sloot. 2017. “Models of pedestrian adaptive behavior in hot outdoor public spaces.” In Vol. 108 of Proc., Int. Conf. on Computational Science (ICCS 2017), 185–194. Amsterdam, Netherlands: Procedia Computer Science. https://doi.org/10.1016/j.procs.2017.05.006.
Oh, S. I., and H. B. Kang. 2017. “Multiple objects fusion tracker using a matching network for adaptively represented instance pairs.” Sensors 17 (4): 883. https://doi.org/10.3390/s17040883.
Ouyang, W., et al. 2015. “DeepID-Net: Deformable deep convolutional neural networks for object detection.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). New York: IEEE.
Paisitkriangkrai, S., C. Shen, and A. van den Hengel. 2015. “Pedestrian detection with spatially pooled features and structured ensemble learning.” IEEE Trans. Pattern Anal. Mach. Intell. 38 (6): 1243–1257. https://doi.org/10.1109/TPAMI.2015.2474388.
Peng, C., T. Xiao, Z. Li, Y. Jiang, X. Zhang, K. Jia, G. Yu, and J. Sun. 2018. “MegDet: A large mini-batch object detector.” In Proc., IEEE/CVF Conf. on Computer Vision and Pattern Recognition. New York: IEEE.
Redmon, J., S. Divvala, R. Girshick, and A. Farhadi. 2016. “You only look once: Unified, real-time object detection.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition. New York: IEEE.
Redmon, J., and A. Farhadi. 2017. “YOLO9000: Better, faster, stronger.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). New York: IEEE.
Redmon, J., and A. Farhadi. 2018. “YOLOv3: An Incremental Improvement.” Preprint, submitted April 8, 2018. http://arxiv.org/abs/1804.02767.
Ren, S., K. He, R. Girshick, and J. Sun. 2016. “Faster R-CNN: Towards real-time object detection with region proposal networks.” IEEE Trans. Pattern Anal. Mach. Intell. 39 (6): 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031.
Soon, F. C., H. Y. Khaw, J. H. Chuah, and J. Kanesan. 2018. “Hyper-parameters optimisation of deep CNN architecture for vehicle logo recognition.” IET Intel. Transport Syst. 12 (8): 939–946. https://doi.org/10.1049/iet-its.2018.5127.
Springstein, M., and R. Ewerth. 2016. “On the effects of spam filtering and incremental learning for web-supervised visual concept classification.” In Proc., 2016 ACM Int. Conf. on Multimedia Retrieval, 377–380.
Taigman, Y., M. Yang, M. Ranzato, and L. Wolf. 2014. “DeepFace: Closing the gap to human-level performance in face verification.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition. New York: IEEE.
Tang, S., M. Andriluka, and B. Schiele. 2014. “Detection and tracking of occluded people.” Int. J. Comput. Vision 110 (1): 58–69. https://doi.org/10.1007/s11263-013-0664-6.
Tourani, A., S. Soroori, A. Shahbahrami, S. Khazaee, and A. Akoushideh. 2019. “A robust vehicle detection approach based on faster R-CNN algorithm.” In Proc., 4th Int. Conf. on Pattern Recognition and Image Analysis (IPRIA), 119–123. Piscataway, NJ: IEEE. https://doi.org/10.1109/PRIA.2019.8785988.
Ucar, A., Y. Demir, and C. Guzelis. 2016. “Moving towards in object recognition with deep learning for autonomous driving applications.” In Proc., 2016 Int. Symp. on Innovations in Intelligent Systems and Applications (INISTA). Piscataway, NJ: IEEE. https://doi.org/10.1109/INISTA.2016.7571862.
Viola, P., and M. Jones. 2001. “Rapid object detection using a boosted cascade of simple features.” In Proc., 2001 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition. New York: IEEE.
Wang, J., M. Chen, W. Yan, Y. Zhi, and Z. Wang. 2019. “A data-driven approach to estimate the probability of pedestrian flow congestion at transportation bottlenecks.” KSCE J. Civ. Eng. 23 (1): 251–259. https://doi.org/10.1007/s12205-018-0063-1.
Xie, W., E. W. M. Lee, Y. Cheng, M. Shi, R. Cao, and Y. Zhang. 2020. “Evacuation performance of individuals and social groups under different visibility conditions: Experiments and surveys.” Int. J. Disaster Risk Reduct. 47 (Aug): 101527. https://doi.org/10.1016/j.ijdrr.2020.101527.
Yogameena, B., and C. Nagananthini. 2017. “Computer vision based crowd disaster avoidance system: A survey.” Int. J. Disaster Risk Reduct. 22 (Jun): 95–129. https://doi.org/10.1016/j.ijdrr.2017.02.021.
Zanlungo, F., Z. Yucel, D. Brscic, T. Kanda, and N. Hagita. 2017. “Intrinsic group behavior : Dependence of pedestrian dyad dynamics on principal social and personal features.” PLoS One 12 (11): e0187253. https://doi.org/10.1371/journal.pone.0187253.
Zhang, S., R. Benenson, and B. Schiele. 2015. “Filtered channel features for pedestrian detection.” In Proc., 2015 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 1751–1760. Piscataway, NJ: IEEE. https://doi.org/10.1109/CVPR.2015.7298784.
Zhang, Z., M. Li, X. Lin, and Y. Wang. 2020. “Network-wide traffic flow estimation with insufficient volume detection and crowdsourcing data.” Transp. Res. Part C Emerging Technol. 121 (Dec): 102870. https://doi.org/10.1016/j.trc.2020.102870.
Zhao, X., R. Lovreglio, and D. Nilsson. 2020. “Modelling and interpreting pre-evacuation decision-making using machine learning.” Autom. Constr. 113 (May): 103140. https://doi.org/10.1016/j.autcon.2020.103140.
Zheng, X., H. Y. Li, L. Y. Meng, X. Y. Xu, and Y. H. Yang. 2017. “Simulating queuing behavior of pedestrians in subway stations.” Proc. Inst. Civ. Eng. Transp. 170 (6): 373–380. https://doi.org/10.1680/jtran.16.00084.
Zhou, Z. B., S. Mo, S. Du, and J. Geng. 2017. “Behaviors recognition and step detection for pedestrian navigation via a foot-mounted inertial measurement unit.” In Proc., 30th Int. Technical Meeting of the Satellite Division of the Institute of Navigation (ION GNSS 2017), 357–367. San Diego: Institute of Navigation. https://doi.org/10.33012/2017.15194.
Information & Authors
Information
Published In
Journal of Transportation Engineering, Part A: Systems
Volume 149 • Issue 2 • February 2023
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Nov 17, 2021
Accepted: Aug 31, 2022
Published online: Dec 13, 2022
Published in print: Feb 1, 2023
Discussion open until: May 13, 2023
ASCE Technical Topics:
- Algorithms
- Business management
- Design (by type)
- Engineering fundamentals
- Human and behavioral factors
- Infrastructure
- Mathematics
- Model accuracy
- Models (by type)
- Multiscale methods
- Pedestrians and cyclists
- Practice and Profession
- Public administration
- Public health and safety
- Safety
- Structural design
- Traffic engineering
- Transportation engineering
- Urban and regional development
- Urban areas
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Lili Meng, Xi Cui, Ran Liu, Zhi Zheng, Hongli Shao, Jinxiang Liu, Yao Peng, Lei Zheng, Research on Metallurgical Saw Blade Surface Defect Detection Algorithm Based on SC-YOLOv5, Processes, 10.3390/pr11092564, 11, 9, (2564), (2023).
- Ayush Shah, Dhruv Kudalkar, Jash Shah, Neha Katre, Proposed Methodology for Real-Time Pistol Detection System, 2023 International Conference on Advanced Computing Technologies and Applications (ICACTA), 10.1109/ICACTA58201.2023.10393546, (1-6), (2023).