A Novel Lightweight Traffic Sign Recognition Model Based on YOLOv5
Publication: Journal of Transportation Engineering, Part A: Systems
Volume 149, Issue 5
Abstract
How to strike a balance between detection speed and recognition accuracy has become a major challenge in real-time object detection. In this research, the YOLOv5 (You Only Look Once version 5) model was lightweight and optimized to improve the detection speed and accuracy of the network. To prune the backbone and neck are to simplify the network structure and reduce the parameters. The lightweight structure of the C3 module was designed and incorporated into the attention mechanism to improve the feature extraction capability of the network. For the public traffic sign dataset, the label assignment strategy and the loss function of YOLOv5 were refined to alleviate the imbalance between positive and negative samples and to better compute the loss, resulting in more stable and efficient training. Compared with other mainstream single-stage models, it achieves a better trade-off between speed and accuracy. With only 0.85 M parameters, 91.9% of mAP (mean average precision) and 360 FPS (frames per second) were achieved, which were 16.26% mAP and 26.67 FPS higher than the conventional YOLOv5n, respectively. The performance of our lightweight model in traffic sign detection far exceeds the most advanced achievements.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, and software code generated or used during the study appear in the published article.
Acknowledgments
We are grateful to Prof. Wenju Li for his patient guidance. This work was supported by the Natural Science Foundation of China (No. 61903256).
References
Alam, A., and Z. A. Jaffery. 2020. “Indian traffic sign detection and recognition.” Int. J. Intell. Syst. 18 (1): 98–112. https://doi.org/10.1007/s13177-019-00178-1.
Andrew, A. M. 2001. “An introduction to support vector machines and other kernel-based learning methods.” Kybernetes 30 (1): 103–115. https://doi.org/10.1108/k.2001.30.1.103.6.
Bai, J., and C. Sun. 2019. “Research on traffic sign detection algorithms based on HSV color model and shape characteristics.” J. Henan Sci. Technol. 2019 (23): 91–92. https://doi.org/10.3969/j.issn.1003-5168.2019.23.033.
Batool, A., M. W. Nisar, J. H. Shah, M. A. Khan, and A. A. El-Latif. 2022. iELMNet: Integrating novel improved extreme learning machine and convolutional neural network model for traffic sign detection. New Rochelle, NY: Mary Ann Liebert Inc.
Bochkovskiy, A., C.-Y. Wang, and H.-Y. M. Liao. 2020. Yolov4: Optimal speed and accuracy of object detection. New York: IEEE.
Cai, J., H. Qiu, J. Tan, and P. Yang. 2022. “Research on traffic sign recognition algorithm based on multi-scale context fusion.” Radio Eng. 52 (1): 114–120.
Chen, C., H. Wang, Y. Zhao, Y. Wang, L. Li, K. Li, and T. Zhang. 2021. “A novel traffic sign recognition algorithm based on deep learning.” Telecommun. Eng. 61 (1): 76–82. https://doi.org/10.3969/j.issn.1001-893x.2021.01.012.
Chen, M., and S. Yu. 2022. “Research on traffic sign recognition based on improved YOLOV4 Model [J/OL]. Microelectronics and computers: 1–10.” Microelectr. Comput. 39 (1): 17–25. https://doi.org/10.19304/J.ISSN1000-7180.2021.0858.
Chollet, F. 2017. “Xception: Deep learning with depthwise separable convolutions.” In Proc., 2017 IEEE Conf. on Computer Vision and Pattern Recognition, 1800–1807. New York: IEEE.
Du, Y., Y. Jia, and J. Han. 2018. “A detection and recognition method for traffic speed limit signs based on vehicle videos.” J. Geom. 43 (2): 32–34. https://doi.org/10.14188/j.2095-6045.2018018.
Fleyeh, H., R. Biswas, and E. Davami. 2013. Traffic sign detection based on AdaBoost color segmentation and SVM classification, 2005–2010. New York: IEEE.
Ge, Z., S. Liu, Z. Li, O. Yoshie, and J. Sun. 2021a. “OTA: Optimal transport assignment for object detection.” In Proc., 2021 IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 303–312. New York: IEEE.
Ge, Z., S. Liu, F. Wang, Z. Li, and J. Sun. 2021b. “Yolox: Exceeding yolo series in 2021.” Preprint, submitted July 18, 2021. https://arxiv.org/abs/2107.08430.
Gong, Y. L., Y. Wu, and M. Z. Chen. 2019. “An enlargement strategy for TT100K traffic sign data set.” J. Fujian Comput. 35 (11): 70–71. https://doi.org/10.16707/j.cnki.fjpc.2019.11.023.
Greenhalgh, J., and M. Mirmehdi. 2012. “Real-time detection and recognition of road traffic signs.” IEEE Trans. Intell. Transp. Syst. 13 (4): 1498–1506. https://doi.org/10.1109/TITS.2012.2208909.
Han, K., Y. Wang, Q. Tian, J. Guo, C. Xu, and C. Xu. 2020. “GhostNet: More features from cheap operations.” In Proc., 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 1577–1586. New York: IEEE.
Haque, W. A., S. Arefin, and A. S. M. Shihavuddin. 2021. “DeepThin: A novel lightweight CNN architecture for traffic sign recognition without GPU requirements.” Expert Syst. Appl. 168 (Apr): 114481. https://doi.org/10.1016/j.eswa.2020.114481.
He, K., X. Zhang, and S. Ren. 2015. “Spatial pyramid pooling in deep convolutional networks for visual recognition.” IEEE Trans. Pattern Anal. Mach. Intell. 37 (9): 1904–1916. https://doi.org/10.1109/TPAMI.2015.2389824.
He, S., L. Chen, S. Zhang, Z. Guo, P. Sun, H. Liu, and H. Liu. 2021. “Automatic recognition of traffic signs based on visual inspection.” IEEE Access 9 (21): 43253–43261. https://doi.org/10.1109/ACCESS.2021.3059052.
Howard, A., et al. 2019. “Searching for MobileNetV3.” In Proc., 2019 IEEE/CVF Int. Conf. on Computer Vision, 1314–1324. New York: IEEE.
Howard, A. G., M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam. 2017. “Mobilenets: Efficient convolutional neural networks for mobile vision applications.” Preprint, submitted April 17, 2017. https://arxiv.org/abs/1704.04861.
Jiang, B., R. Luo, J. Mao, T. Xiao, and Y. Jiang. 2018. “Acquisition of localization confidence for accurate object detection.” In Proc., European Conf. on Computer Vision (ECCV), 784–799. Berlin: Springer.
Jin, Y., Y. Fu, W. Wang, J. Guo, C. Ren, and X. Xiang. 2020. “Multi-feature fusion and enhancement single shot detector for traffic sign recognition.” IEEE Access 8 (65): 38931–38940. https://doi.org/10.1109/ACCESS.2020.2975828.
Jocher, G., A. Stoken, A. Chaurasia, J. Borovec, T. Xie, Y. Kwon, K. Michael, L. Changyu, J. Fang, and V. Abrahim. 2021. Ultralytics/yolov5: v6. 0—YOLOv5n “Nano” models, Roboflow integration, tensor, flow export, open, CV DNN support. Honolulu, HI: Zenodo.
Kiruthika Devi, S., and C. N. Subalalitha. 2022. “A deep learning–based residual network model for traffic sign detection and classification.” In Ubiquitous intelligent systems, 71–83. Berlin: Springer.
Li, X., Z. Xie, X. Deng, Y. Wu, and Y. Pi. 2022. “Traffic sign detection based on improved faster R-CNN for autonomous driving.” J. Supercomput. 2022 (1): 1–21. https://doi.org/10.1007/s11227-021-04230-4.
Liang, M., X. Cui, Q. Song, and X. Zhao. 2017. “Traffic sign recognition method based on HOG-Gabor feature fusion and Softmax classifier.” J. Traffic Transp. Eng. 17 (3): 151–158. https://doi.org/10.3969/j.issn.1671-1637.2017.03.016.
Lin, T. Y., P. Goyal, R. Girshick, K. He, and P. Dollár. 2017. “Focal loss for dense object detection.” In Proc., 2017 IEEE Int. Conf. on Computer Vision, 2999–3007. New York: IEEE.
Liu, S., L. Qi, and H. Qin. 2018. “Path aggregation network for instance segmentation.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 8759–8768. New York: IEEE.
Ma, N., X. Zhang, H.-T. Zheng, and J. Sun. 2018. “Shufflenet v2: Practical guidelines for efficient CNN architecture design.” In Proc., European Conf. on Computer Vision, 116–131. Berlin: Springer.
Mangshor, N. N. A., N. P. A. M. Paudzi, S. Ibrahim, and N. Sabri. 2022. “A real-time Malaysian traffic sign recognition using YOLO algorithm.” In Proc., 12th National Technical Seminar on Unmanned System Technology, 283–293. Berlin: Springer.
Nadeem, Z., Z. Khan, U. Mir, U. I. Mir, S. Khan, H. Nadeem, and J. Sultan. 2022. “Pakistani traffic-sign recognition using transfer learning.” Multimed. Tools Appl. 81 (6): 8429–8449. https://doi.org/10.1007/s11042-022-12177-8.
Pan, W., B. Liu, Y. Chen, and H. Shi. 2019. “Traffic sign detection and recognition based on YOLO v3.” Transduction Microsyst. Technol. 38 (11): 147–150. https://doi.org/10.13873/j.1000-9787(2019)11-0147-04.
Redmon, J., and A. Farhadi. 2018. “Yolov3: An incremental improvement.” Preprint, submitted April 8, 2018. https://arxiv.org/abs/1804.02767.
Ren, S., K. He, R. Girshick, and J. Sun. 2015. “Faster r-cnn: Towards real-time object detection with region proposal networks.” Adv. Neural Inf. Process. Syst. 2015 (1): 28.
Sandler, M., A. Howard, M. Zhu, A. Zhmoginov, and L. C. Chen. 2018. “MobileNetV2: Inverted residuals and linear bottlenecks.” In Proc., 2018 IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 4510–4520. New York: IEEE.
Sanyal, B., R. Kumar Mohapatra, and R. Dash. 2022. “Traffic sign recognition on Indian database using wavelet descriptors and convolutional neural network ensemble.” Concurr. Comput. Pract. Exp. 2022 (1): e6827. https://doi.org/10.1002/cpe.6827.
Singh, K., and N. Malik. 2022. “CNN based approach for traffic sign recognition system.” Adv. J. Grad. Res. 11 (1): 23–33. https://doi.org/10.21467/ajgr.11.1.23-33.
Stallkamp, J., M. Schlipsing, and J. Salmen. 2012. “Man versus computer: Benchmarking machine learning algorithms for traffic sign recognition.” Neural Networks 32 (2): 323–332. https://doi.org/10.1016/j.neunet.2012.02.016.
Stallkamp, J., M. Schlipsing, J. Salmen, and C. Igel. 2011. “The German traffic sign recognition benchmark: A multi-class classification competition.” In Proc., 2011 Int. Joint Conf. on Neural Networks, 1453–1460. New York: IEEE.
Tan, M., R. Pang, and Q. V. Le. 2020. “EfficientDet: Scalable and efficient object detection.” In Proc., 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 10778–10787. New York: IEEE.
Tang, J., and Q. Li. 2010. “Fast template matching algorithm.” J. Comput. Appl. 236 (6): 1559–1561. https://doi.org/10.1016/j.cam.2011.09.002.
Tang, Q., G. Cao, and K. H. Jo. 2021. “Integrated feature pyramid network with feature aggregation for traffic sign detection.” IEEE Access 9 (12): 117784–117794. https://doi.org/10.1109/ACCESS.2021.3106350.
Tian, Z., C. Shen, H. Chen, and T. He. 2019. “FCOS: Fully convolutional one-stage object detection.” In Proc., 2019 IEEE/CVF Int. Conf. on Computer Vision (ICCV), 9626–9635. New York: IEEE.
Wang, C. Y., H. Y. M. Liao, and Y. H. Wu. 2020. “CSPNet: A new backbone that can enhance learning capability of CNN.” In Proc., IEEE/CVF Conf. on Computer Vision and Pattern Recognition Workshops, 390–391. New York: IEEE.
Wei, T., X. Chen, and Y. Yin. 2021. “Research on traffic sign recognition method based on multi-scale convolution neural network.” J. Northwestern Polytech. Univ. 39 (4): 891–900. https://doi.org/10.1051/jnwpu/20213940891.
Woo, S., J. Park, J.-Y. Lee, and I. S. Kweon. 2018. “Cbam: Convolutional block attention module.” In Proc., European Conf. on Computer Vision, 3–19. Berlin: Springer.
Wu, B., A. Wan, X. Yue, P. Jin, S. Zhao, N. Golmant, A. Gholaminejad, J. Gonzalez, and K. Keutzer. 2018. “Shift: A zero FLOP, zero parameter alternative to spatial convolutions.” In Proc., 2018 IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 9127–9135. New York: IEEE.
Xu, K., J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhudinov, R. Zemel, and Y. Bengio. 2015. “Show, attend and tell: Neural image caption generation with visual attention.” Preprint, submitted February 10, 2015. https://arxiv.org/abs/1502.030441804.02767.
Youssouf, N. 2022. Traffic sign detection and recognition with faster-RCNN and YOLOV4. Rochester, NY: Social Science Electronic Publishing.
Yu, G., Q. Chang, W. Lv, C. Xu, C. Cui, W. Ji, Q. Dang, K. Deng, G. Wang, and Y. Du. 2021. “PP-PicoDet: A better real-time object detector on mobile devices.” Preprint, submitted November 1, 2021. https://arxiv.org/abs/2111.00902.
Zaklouta, F., B. Stanciulescu, and O. Hamdoun. 2011. “Traffic sign classification using K-d trees and random forests.” In Proc., 2011 Int. Joint Conf. on Neural Networks, 2151–2155. New York: IEEE.
Zanchettin, C., and P. Novais. 2021. “Construction of Brazilian regulatory traffic sign recognition dataset.” In Proc., Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications: 25th Iberoamerican Congress. Berlin: Springer.
Zeng, H. 2022. “Real-time traffic sign detection based on improved YOLO V3.” In Proc., 11th Int. Conf. on Computer Engineering and Networks, 167–172. Berlin: Springer.
Zhang, H., Y. Wang, F. Dayoub, and N. Sünderhauf. 2021. “VarifocalNet: An IoU-aware dense object detector.” In Proc., 2021 IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 8510–8519. Berlin: IEEE.
Zhang, J.-D., X.-B. Xu, L.-B. Lu, and Y.-Q. Zhao. 2022. “Research on traffic sign recognition method based on deep residual network.” Comput. Simul. 39 (1): 143–147. https://doi.org/10.3969/j.issn.1006-9348.2022.01.031.
Zhang, X., X. Zhou, M. Lin, and J. Sun. 2018. “ShuffleNet: An extremely efficient convolutional neural network for mobile devices.” In Proc., 2018 IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 6848–6856. New York: IEEE.
Zheng, Z., P. Wang, W. Liu, J. Li, R. Ye, and D. Ren. 2020. “Distance-IoU loss: Faster and better learning for bounding box regression.” In Proc., AAAI Conf. on Artificial Intelligence, 12993–13000. Menlo Park, CA: AAAI Press.
Zhu, Z., D. Liang, S. Zhang, X. Huang, and S. Hu. 2016a. “Traffic-sign detection and classification in the wild.” Tsinghua University-Tencent Joint Laboratory. Accessed December 25, 2020. http://cg.cs.tsinghua.edu.cn/traffic-sign/.
Zhu, Z., D. Liang, S. Zhang, X. Huang, B. Li, and S. Hu. 2016b. “Traffic-sign detection and classification in the wild.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 2110–2118. New York: IEEE.
Information & Authors
Information
Published In
Journal of Transportation Engineering, Part A: Systems
Volume 149 • Issue 5 • May 2023
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Apr 2, 2022
Accepted: Nov 9, 2022
Published online: Feb 24, 2023
Published in print: May 1, 2023
Discussion open until: Jul 24, 2023
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Jiquan Shen, Ziyang Zhang, Junwei Luo, Xiaohong Zhang, YOLOv5-TS: Detecting traffic signs in real-time, Frontiers in Physics, 10.3389/fphy.2023.1297828, 11, (2023).
- Pang Jo Chun, M. Suzuki, Y. Kato, Iterative application of generative adversarial networks for improved buried pipe detection from images obtained by ground‐penetrating radar, Computer-Aided Civil and Infrastructure Engineering, 10.1111/mice.13070, 38, 17, (2472-2490), (2023).