Crack Detection and Measurement Using PTZ Camera–Based Image Processing Method on Expressways
Publication: Journal of Transportation Engineering, Part B: Pavements
Volume 148, Issue 3
Abstract
Cracks commonly develop in expressways owing to repeated vehicle loads and environmental impact. Therefore, timely and accurate detection of minor cracks can help prevent severe and expensive rehabilitation efforts as well as assist in inspections to avert significant transportation system failures. Consequently, crack detection and measurement in expressways has been extensively researched. Traditional two-dimensional approaches present challenges, as they typically require distance measurement data and large workload. A pan–tilt–zoom (PTZ) camera–based crack detection and measurement method based on deep learning to obtain the size measurement of detected cracks is proposed in this paper. A YOLOv5-based crack recognition model is first trained to detect and classify crack images captured by PTZ cameras. Our previously proposed crack size estimation method is then applied on these crack images to estimate length and width. The proposed approach successfully detects and measures cracks on highway pavements; the YOLOv5-based crack recognition results yield 87.7% mean average precision and more than 90% average precision for crack real size measurement on 12 cracks. Case studies conducted in the G4/Jingshi Highway demonstrate the applicability of the method and enable tuning of the measurement algorithm parameters, confirming the viability of our proposal.
Practical Applications
Cracks in highway pavement are among the most critical problems in expressway maintenance. Crack detection and measurement not only have severe consequences in relation to expressway safety and decide maintenance timing, but can also prevent the rapid development of crack disease. Owing to ITS development of an expressway network in China, PTZ camera–based crack detection and measurement using YOLOv5 networks is proposed. We train a crack recognition model using YOLOv5 to classify cracks before length and width estimation. Case studies conducted in the G4/Jingshi Highway in the Hebei province of China demonstrate that the proposed method was able to detect and quantify cracks in a cost-effective manner, providing objective and timely guidance for preventive maintenance. This method shows its promise in real-world civil engineering applications for network-level highway maintenance engineering.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, and code generated or used during the study appear in the published article.
Acknowledgments
This work was supported by Hainan Provincial Natural Science Foundation of China (No. 521QN206) and Scientific Research Foundation for Hainan University (No. KYQD(ZR)-21013).
References
Ammar, A., A. Koubaa, M. Ahmed, and A. Saad. 2019. Aerial images processing for car detection using convolutional neural networks: Comparison between faster r-CNN and YOLOv3. Basel, Switzerland: Multidisciplinary Digital Publishing Institute.
Barner, K. E., A. M. Sarhan, and R. C. Hardie. 1999. “Partition-based weighted sum filters for image restoration.” IEEE Trans. Image Process. 8 (5): 740–745. https://doi.org/10.1109/83.760341.
Benjdira, B., T. Khursheed, A. Koubaa, A. Ammar, and K. Ouni. 2019. “Car detection using unmanned aerial vehicles: Comparison between faster r-CNN and YOLOv3.” In Proc., 2019 1st Int. Conf. on Unmanned Vehicle Systems-Oman (UVS), 1–6. Piscataway, NJ: IEEE.
Dadrasjavan, F., N. Zarrinpanjeh, and A. Ameri. 2019. Automatic crack detection of road pavement based on aerial UAV imagery. Basel, Switzerland: Multidisciplinary Digital Publishing Institute.
Fan, R., M. J. Bocus, Y. Zhu, J. Jiao, L. Wang, F. Ma, S. Cheng, and M. Liu. 2019. “Road crack detection using deep convolutional neural network and adaptive thresholding.” In Proc., 2019 IEEE Intelligent Vehicles Symp. (IV), 474–479. New York: IEEE.
Fan, Z., Y. Wu, J. Lu, and W. Li. 2018. “Automatic pavement crack detection based on structured prediction with the convolutional neural network.” Preprint, submitted February 1, 2018. https://doi.org/10.48550/arXiv.1802.02208.
Huang, J., W. Liu, and X. Sun. 2014. “A pavement crack detection method combining 2D with 3D information based on Dempster-Shafer theory.” Comput.-Aided Civ. Infrastruct. Eng. 29 (4): 299–313. https://doi.org/10.1111/mice.12041.
Huang, L., W. Zhao, Z. Sun, and J. Wang. 2015. “An advanced gradient histogram and its application for contrast and gradient enhancement.” J. Visual Commun. Image Represent. 31 (Aug): 86–100. https://doi.org/10.1016/j.jvcir.2015.06.007.
Iyer, S., and S. K. Sinha. 2005. “A robust approach for automatic detection and segmentation of cracks in underground pipeline images.” Image Vision Comput. 23 (10): 921–933. https://doi.org/10.1016/j.imavis.2005.05.017.
Kang, D., S. S. Benipal, D. L. Gopal, and Y.-J. Cha. 2020. “Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning.” Autom. Constr. 118 (Oct): 103291. https://doi.org/10.1016/j.autcon.2020.103291.
Lee, B. J., and H. D. Lee. 2004. “Position-invariant neural network for digital pavement crack analysis.” Comput.-Aided Civ. Infrastruct. Eng. 19 (2): 105–118. https://doi.org/10.1111/j.1467-8667.2004.00341.x.
Li, J., G. Liu, T. Yang, J. Zhou, and Y. Zhao. 2019. “Research on relationships among different distress types of asphalt pavements with semi-rigid bases in china using association rule mining: A statistical point of view.” Adv. Civ. Eng. 2019 (1): 1–19. https://doi.org/10.1155/2019/5369532.
Lins, R. G., and S. N. Givigi. 2016. “Automatic crack detection and measurement based on image analysis.” IEEE Trans. Instrum. Meas. 65 (3): 583–590. https://doi.org/10.1109/TIM.2015.2509278.
Nayyeri, F., L. Hou, J. Zhou, and H. Guan. 2019. “Foreground–background separation technique for crack detection.” Comput.-Aided Civ. Infrastruct. Eng. 34 (6): 457–470. https://doi.org/10.1111/mice.12428.
Oliveira, H., and P. L. Correia. 2012. “Automatic road crack detection and characterization.” IEEE Trans. Intell. Transp. Syst. 14 (1): 155–168. https://doi.org/10.1109/TITS.2012.2208630.
Peng, L., W. Chao, L. Shuangmiao, and F. Baocai. 2015. “Research on crack detection method of airport runway based on twice-threshold segmentation.” In Proc., 5th Int. Conf. on Instrumentation and Measurement, Computer, Communication and Control (IMCCC), 1716–1720. New York: IEEE.
Shan, B., S. Zheng, and J. Ou. 2016. “A stereovision-based crack width detection approach for concrete surface assessment.” KSCE J. Civ. Eng. 20 (2): 803–812. https://doi.org/10.1007/s12205-015-0461-6.
Shao, C., Q. Ding, H. Luo, Z. Chang, C. Zhang, and T. Zheng. 2017. “Step-by-step pipeline processing approach for line segment detection.” IET Image Proc. 11 (6): 416–424. https://doi.org/10.1049/iet-ipr.2016.0493.
Shao, C., Y. Du, Q. Du, and P. Ning. 2020. “PTZ camera–based image processing pipeline for automatic size estimation of cracks in expressways.” In Proc., 2020 Transportation Research Board 99th Annual Meeting (TRB). Washington, DC: Transportation Research Board.
Shao, C., L. Zhang, and W. Pan. 2021. “PTZ camera–based image processing for automatic crack size measurement in expressways.” IEEE Sens. J. 21 (20): 23352–23361. https://doi.org/10.1109/JSEN.2021.3112005.
Vaitkus, A., A. Laurinavičius, R. Oginskas, A. Motiejūnas, M. Paliukaitė, and O. Barvidienė. 2012. “The road of experimental pavement structures: Experience of five years operation.” Balt. J. Road Bridge Eng. 7 (3): 220–227. https://doi.org/10.3846/bjrbe.2012.30.
Wang, S., and W. Tang. 2011. “Pavement crack segmentation algorithm based on local optimal threshold of cracks density distribution.” In Proc., Int. Conf. on Intelligent Computing, 298–302. Berlin: Springer.
Wang, X., and Z. Hu. 2017. “Grid-based pavement crack analysis using deep learning.” In Proc., 2017 4th Int. Conf. on Transportation Information and Safety (ICTIS), 917–924. New York: IEEE.
Yan, B., P. Fan, X. Lei, Z. Liu, and F. Yang. 2021. “A real-time apple targets detection method for picking robot based on improved YOLOv5.” Remote Sens. 13 (9): 1619. https://doi.org/10.3390/rs13091619.
Yang, Y.-S., C.-M. Yang, and C.-W. Huang. 2015. “Thin crack observation in a reinforced concrete bridge pier test using image processing and analysis.” Adv. Eng. Software 83 (5): 99–108. https://doi.org/10.1016/j.advengsoft.2015.02.005.
Yi, S., R. M. Haralick, and L. G. Shapiro. 1995. “Optimal sensor and light source positioning for machine vision.” Comput. Vision Image Understanding 61 (1): 122–137. https://doi.org/10.1006/cviu.1995.1009.
Ying, L., and E. Salari. 2010. “Beamlet transform-based technique for pavement crack detection and classification.” Comput.-Aided Civ. Infrastruct. Eng. 25 (8): 572–580. https://doi.org/10.1111/j.1467-8667.2010.00674.x.
Zhang, A., K. C. P. Wang, Y. Fei, Y. Liu, S. Tao, C. Chen, J. Q. Li, and B. Li. 2018. “Deep learning–based fully automated pavement crack detection on 3D asphalt surfaces with an improved CrackNet.” J. Comput. Civ. Eng. 32 (5): 04018041. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000775.
Zou, Q., Y. Cao, Q. Li, Q. Mao, and S. Wang. 2012. “Cracktree: Automatic crack detection from pavement images.” Pattern Recognit. Lett. 33 (3): 227–238. https://doi.org/10.1016/j.patrec.2011.11.004.
Information & Authors
Information
Published In
Journal of Transportation Engineering, Part B: Pavements
Volume 148 • Issue 3 • September 2022
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Apr 19, 2021
Accepted: May 3, 2022
Published online: Jul 5, 2022
Published in print: Sep 1, 2022
Discussion open until: Dec 5, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Zhiqiang Xia, Chunyan Shao, Xuezhi Feng, Huaiqiang Wang, Step‐by‐step image enhancement method for PTZ‐camera based crack detection in expressways, Electronics Letters, 10.1049/ell2.13098, 60, 3, (2024).