Measurement of Asphalt Pavement Crack Length Using YOLO V5-BiFPN
Publication: Journal of Infrastructure Systems
Volume 30, Issue 2
Abstract
Pavement cracks are a kind of common distress in road service time, and their length measurement is critical for pavement maintenance. The current automatic method of crack length measurement uses segmentation algorithms to obtain crack curves, which is time-consuming and complex. In this study, an effective method of crack length measurement was proposed and validated. The method consists of a detection module based on an object detection algorithm and a length calculation module. To increase the speed and accuracy of crack detection, an improved pavement crack detection algorithm BiFPN-enhanced YOLO V5 (YOLO V5-BiFPN) based on you look only once version 5 (YOLO V5) and bidirectional feature pyramid network (BiFPN) is proposed, and gamma correction was utilized to process pavement images. YOLO V5-BiFPN was tested in a real pavement image data set and achieved remarkable performance. In the length calculation module, the diagonal length of the crack bounding box output by the object detection algorithm can be defined as the crack length. To validate the measurement method, the true value of crack length was obtained from the segmentation data set by skeletonization. The error between the calculation result of the proposed method and the real value is 3.4%, and the average processing time of each image is 14.2 ms. The developed method addresses the problem of considerable time and financial cost associated with the existing crack length measurement methods.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This study is sponsored by the projects found by the National Natural Science Foundation of China (NSFC) under Grant Nos. 51978163 and 52208439, the Jiangsu Nature Science Foundation under Grant No. BK20200468, to which the authors are very grateful.
References
Bang, S., S. Park, H. Kim, and H. Kim. 2019. “Encoder-decoder network for pixel-level road crack detection in black-box images.” Comput.-Aided Civ. Infrastruct. Eng. 34 (8): 713–727. https://doi.org/10.1111/mice.12440.
Bochkovskiy, A., C.-Y. Wang, and H.-Y. M. Liao. 2020. “Yolov4: Optimal speed and accuracy of object detection.” Preprint, submitted April 23, 2020. http://arxiv.org/abs/2004.10934.
Bull, D. R. 2014. “Chapter 4—Digital Picture formats and representations.” In Communicating pictures, edited by D. R. Bull, 99–132. Oxford, UK: Academic Press. https://doi.org/10.1016/B978-0-12-405906-1.00004-0.
Cao, J.-N., S.-S. Xu, and C.-Q. Li. 2014. “Automatic road crack identification and its measurement.” Chang’an Daxue Xuebao (Ziran Kexue Ban)/J. Chang’an Univ. 34 (3): 17–22. https://doi.org/10.19721/j.cnki.1671-8879.2014.03.003.
Dasiopoulou, S., V. Mezaris, I. Kompatsiaris, V. K. Papastathis, and M. G. Strintzis. 2005. “Knowledge-assisted semantic video object detection.” IEEE Trans. Circuits Syst. Video Technol. 15 (10): 1210–1224. https://doi.org/10.1109/TCSVT.2005.854238.
Dong, J., N. Wang, H. Fang, Q. Hu, C. Zhang, B. Ma, D. Ma, and H. Hu. 2022. “Innovative method for pavement multiple damages segmentation and measurement by the Road-Seg-CapsNet of feature fusion.” Constr. Build. Mater. 324 (Mar): 126719. https://doi.org/10.1016/j.conbuildmat.2022.126719.
Du, Y. C., N. Pan, Z. H. Xu, F. W. Deng, Y. Shen, and H. Kang. 2021. “Pavement distress detection and classification based on YOLO network.” Int. J. Pavement Eng. 22 (13): 1659–1672. https://doi.org/10.1080/10298436.2020.1714047.
Eisenbach, M., R. Stricker, D. Seichter, K. Amende, K. Debes, M. Sesselmann, D. Ebersbach, U. Stoeckert, and H.-M. Gross. 2017. “How to get pavement distress detection ready for deep learning? A systematic approach.” In Proc., 2017 Int. Joint Conf. on Neural Networks (IJCNN), 2039–2047. New York: IEEE.
Escalona, U., F. Arce, E. Zamora, and H. Sossa. 2019. “Fully convolutional networks for automatic pavement crack segmentation.” Computacion Y Sistemas 23 (2): 451–460. https://doi.org/10.13053/cys-23-2-3047.
Gibb, S., H. M. La, and S. Louis. 2018. “A genetic algorithm for convolutional network structure optimization for concrete crack detection.” In Proc., IEEE Congress on Evolutionary Computation (IEEE CEC) as part of the IEEE World Congress on Computational Intelligence (IEEE WCCI), 1708–1715. New York: IEEE.
Gopalakrishnan, K., S. K. Khaitan, A. Choudhary, and A. Agrawal. 2017. “Deep Convolutional Neural Networks with transfer learning for computer vision-based data-driven pavement distress detection.” Constr. Build. Mater. 157 (Feb): 322–330. https://doi.org/10.1016/j.conbuildmat.2017.09.110.
He, K., X. Zhang, S. Ren, and J. Sun. 2015. “Spatial pyramid pooling in deep convolutional networks for visual recognition.” IEEE Trans. Pattern Anal. Mach. Intell. 37 (9): 1904–1916. https://doi.org/10.1109/TPAMI.2015.2389824.
Hoang, N. D., Q. L. Nguyen, and V. D. Tran. 2018. “Automatic recognition of asphalt pavement cracks using metaheuristic optimized edge detection algorithms and convolution neural network.” Autom. Constr. 94 (Apr): 203–213. https://doi.org/10.1016/j.autcon.2018.07.008.
Hu, G. X., B. L. Hu, Z. Yang, L. Huang, and P. Li. 2021. “Pavement crack detection method based on deep learning models.” Wireless Commun. Mobile Comput. 2021 (May): 1–13. https://doi.org/10.1155/2021/5573590.
Huyan, J., W. Li, S. Tighe, J. Z. Zhai, Z. C. Xu, and Y. Chen. 2019. “Detection of sealed and unsealed cracks with complex backgrounds using I (A) deep convolutional neural network.” Autom. Constr. 107 (Nov): 102946. https://doi.org/10.1016/j.autcon.2019.102946.
Ibragimov, E., H. J. Lee, J. J. Lee, and N. Kim. 2022. “Automated pavement distress detection using region based convolutional neural networks.” Int. J. Pavement Eng. 23 (6): 1981–1992. https://doi.org/10.1080/10298436.2020.1833204.
Jeong, D. J. 2020. “Road damage detection using YOLO with smartphone images.” In Proc., 8th IEEE Int. Conf. on Big Data (Big Data), 5559–5562. New York: IEEE.
Jiang, S., S. Gu, and Z. Yan. 2022. “Pavement crack measurement based on aerial 3D reconstruction and learning-based segmentation method.” Meas. Sci. Technol. 34 (1): 015801. https://doi.org/10.1088/1361-6501/ac8e22.
Ju, H. Y., W. Li, S. S. Tighe, Z. C. Xu, and J. Z. Zhai. 2020. “CrackU-net: A novel deep convolutional neural network for pixelwise pavement crack detection.” Struct. Control Health Monit. 27 (8): e2551. https://doi.org/10.1002/stc.2551.
Liu, S., L. Qi, H. Qin, J. Shi, and J. Jia. 2018. “Path aggregation network for instance segmentation.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 8759–8768. New York: IEEE.
Loshchilov, I., and F. Hutter. 2017. “Decoupled weight decay regularization.” Preprint, Submitted November 14, 2017. https://arxiv.org/abs/1711.05101.
Majidifard, H., P. Jin, Y. Adu-Gyamfi, and W. G. Buttlar. 2020. “Pavement image datasets: A new benchmark dataset to classify and densify pavement distresses.” Transp. Res. Rec. 2674 (2): 328–339. https://doi.org/10.1177/0361198120907283.
Nie, M., and K. Wang. 2018. “Pavement distress detection based on transfer learning.” In Proc., 5th Int. Conf. on Systems and Informatics (ICSAI), 435–439. New York: IEEE.
Poynton, C. 2012. “27—Gamma.” In Digital video and HD, edited by C. Poynton. 2nd ed., 315–334. Boston: Morgan Kaufmann.
Redmon, J., S. Divvala, R. Girshick, and A. Farhadi. 2016. “You only look once: Unified, real-time object detection.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 779–788. New York: IEEE.
Safaei, N., O. Smadi, A. Masoud, and B. Safaei. 2022. “An automatic image processing algorithm based on crack pixel density for pavement crack detection and classification.” Int. J. Pavement Res. Technol. 15 (1): 159–172. https://doi.org/10.1007/s42947-021-00006-4.
Saha, P. K., G. Borgefors, and G. S. di Baja. 2017. Chapter 1—Skeletonization and its applications—A review, 3–42. Cambridge, MA: Academic Press. https://doi.org/10.1016/B978-0-08-101291-8.00002-X.
Tan, M., R. Pang, and Q. V. Le. 2020. “Efficientdet: Scalable and efficient object detection.” In Proc., IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 10781–10790. New York: IEEE.
Tong, Z., J. Gao, Z. Han, and Z. Wang. 2018. “Recognition of asphalt pavement crack length using deep convolutional neural networks.” Road Mater. Pavement Des. 19 (6): 1334–1349. https://doi.org/10.1080/14680629.2017.1308265.
ultralytics. 2022. “yolov5.” Accessed May 30, 2022. https://github.com/ultralytics/yolov5.
Wang, B., X. Wang, F. Chen, Y. He, W. Li, and L. Liu. 2017. “Pavement crack recognition based on aerial image.” Guangxue Xuebao/Acta Opt. Sin. 37 (8): 119–125. https://doi.org/10.3788/AOS201737.0810004.
Wang, C.-Y., H.-Y. M. Liao, Y.-H. Wu, P.-Y. Chen, J.-W. Hsieh, and I.-H. Yeh. 2020. “CSPNet: A new backbone that can enhance learning capability of CNN.” In Proc., IEEE/CVF Conf. on Computer Vision and Pattern Recognition Workshops, 390–391. New York: IEEE.
Wang, W. X., M. F. Wang, H. X. Li, H. Zhao, K. Wang, C. T. He, J. Wang, S. F. Zheng, and J. B. Chen. 2019. “Pavement crack image acquisition methods and crack extraction algorithms: A review.” J. Traffic Transp. Eng. 6 (6): 535–556. https://doi.org/10.1016/j.jtte.2019.10.001.
Wang, X. L., and Z. Z. Hu. 2017. “Grid-based pavement crack analysis using deep learning.” In Proc., 4th Int. Conf. on Transportation Information and Safety (ICTIS), 917–924. New York: IEEE.
Weng, X., Y. Huang, and W. Wang. 2019. “Segment-based pavement crack quantification.” Autom. Constr. 105 (Sep): 102819. https://doi.org/10.1016/j.autcon.2019.04.014.
Xu, Z. C., Z. Y. Sun, J. Huyan, W. Li, and F. P. Wang. 2021. “Pixel-level pavement crack detection using enhanced high-resolution semantic network.” Int. J. Pavement Eng. 23 (14): 4943–4957. https://doi.org/10.1080/10298436.2021.1985491.
Yang, F., L. Zhang, S. J. Yu, D. Prokhorov, X. Mei, and H. B. Ling. 2020. “Feature pyramid and hierarchical boosting network for pavement crack detection.” IEEE Trans. Intell. Transp. Syst. 21 (4): 1525–1535. https://doi.org/10.1109/TITS.2019.2910595.
Yang, X. C., H. Li, Y. T. Yu, X. C. Luo, T. Huang, and X. Yang. 2018. “Automatic pixel-level crack detection and measurement using fully convolutional network.” Comput.-Aided Civ. Infrastruct. Eng. 33 (12): 1090–1109. https://doi.org/10.1111/mice.12412.
Yusof, N. A. M., M. K. Osman, M. H. M. Noor, A. Ibrahim, N. M. Tahir, and N. M. Yusof. 2018. “Crack detection and classification in asphalt pavement images using deep convolution neural network.” In Proc., 8th IEEE Int. Conf. on Control System, Computing and Engineering (ICCSCE), 227–232. New York: IEEE.
Zhang, K. G., H. D. Cheng, and B. Y. Zhang. 2018. “Unified approach to pavement crack and sealed crack detection using preclassification based on transfer learning.” J. Comput. Civ. Eng. 32 (2): 04018001. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000736.
Zhang, Y. F., B. Chen, J. F. Wang, J. M. Li, and X. F. Sun. 2020. “APLCNet: Automatic pixel-level crack detection network based on instance segmentation.” IEEE Access 8 (Aug): 199159–199170. https://doi.org/10.1109/ACCESS.2020.3033661.
Zou, Q., Y. Cao, Q. Q. Li, Q. Z. Mao, and S. Wang. 2012. “Crack tree: Automatic crack detection from pavement images.” Pattern Recognit. Lett. 33 (3): 227–238. https://doi.org/10.1016/j.patrec.2011.11.004.
Information & Authors
Information
Published In
Journal of Infrastructure Systems
Volume 30 • Issue 2 • June 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Jul 11, 2023
Accepted: Feb 7, 2024
Published online: Apr 15, 2024
Published in print: Jun 1, 2024
Discussion open until: Sep 15, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.