PCDNet: Seed Operation–Based Deep Learning Model for Pavement Crack Detection on 3D Asphalt Surface
Publication: Journal of Transportation Engineering, Part B: Pavements
Volume 148, Issue 2
Abstract
The detection of pavement crack plays a critical role in pavement maintenance and rehabilitation because pavement cracking is one of the most important indicators for the pavement condition evaluation, as well as an early manifestation of other pavement distresses. To detect cracks accurately, precisely, and completely based on three-dimensional (3D) pavement images, this paper proposes a deep learning framework based on a convolutional neural network (CNN) and pixel-level improved crack seed algorithm, called Pavement Crack Detection Net (PCDNet). Firstly, the CNN layer based on the convolution implementation of sliding windows is applied to each 3D pavement image to divide it into pavement patches and classify each patch into two types: the background patch, and the pavement crack patch. Secondly, the seed layer, i.e., an automatic threshold pixel-level crack seed recognition algorithm is used to detect the crack distress further and depict the complete contour simultaneously. Finally, the region growing layer is utilized to ensure the continuity of the cracks. Due to the good combination of the CNN and the algorithm, PCDNet needs only a patch-level data set for training but can output pixel-level results, a great novelty in crack detection. In this paper, 5,000 3D pavement images were selected from an established image library. PCDNet was trained with 4,300 3D pavement images and further validated based on 500 3D pavement images. The test experiment based on the remaining 200 images showed that PCDNet can achieve high precision (0.885), recall (0.902), and score (0.893) simultaneously. It also was demonstrated that PCDNet can detect different types of pavement crack under various conditions and resist noncrack pixels with elevation variation features, such as pavement edge drop-offs, curbs, spalling, and bridge expansion joints. Compared with recently developed crack detection methods based on imaging algorithms, PCDNet is capable of not only eliminating more local noise and detecting more fine cracks, but also maintaining much faster processing speed.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some data, models, or codes that support the findings of this study are available from the corresponding author upon reasonable request, including original data of the model test, specific information about the data acquisition equipment, and the demonstration of the model (which automatically can present the result of recognition).
Acknowledgments
This work was supported by the National Key Research and Development Program of China (No. 2017YFC0803902) and the Fundamental Research Funds for the Central Universities. The authors are responsible for all views and opinions expressed in this paper. to the authors thank Prof. Lu’s team for providing pavement 3D data. Funding for this research was provided by the National Key Research and Development Program of China (No. 2017YFC0803902). The authors gratefully acknowledge their support.
References
AASHTO. 2008. Mechanistic-empirical pavement design guide: A manual of practice. Washington, DC: AASHTO.
Amhaz, R., S. Chambon, J. Idier, and V. Baltazart. 2014. “A new minimal path selection algorithm for automatic crack detection on pavement images.” In Proc., 2014 IEEE Int. Conf. on Image Processing (ICIP), 788–792. New York: IEEE. https://doi.org/10.1109/ICIP.2014.7025158.
Bešić, I., N. V. Gestel, J. P. Kruth, P. Bleys, and J. Hodoli. 2011. “Accuracy improvement of laser line scanning for feature measurements on CMM.” Opt. Laser Eng. 49 (11): 1274–1280. https://doi.org/10.1016/j.optlaseng.2011.06.009.
Cha, Y.-J., W. Choi, and O. Buyukozturk. 2017. “Deep learning-based crack damage detection using convolutional neural networks.” Comput.-Aided Civ. Infrastruct. Eng. 32 (5): 361–378. https://doi.org/10.1111/mice.12263.
Chen, F.-C., and M. R. Jahanshahi. 2017. “NB-CNN: Deep learning-based crack detection using convolutional neural network and naïve Bayes data fusion.” IEEE Trans. Ind. Electron. 65 (5): 4392–4400. https://doi.org/10.1109/TIE.2017.2764844.
Fei, Y., K. C. P. Wang, A. Zhang, C. Chen, J. Q. Li, Y. Liu, G. Yang, and B. Li. 2020. “Pixel-level cracking detection on 3D asphalt pavement images through deep-learning-based CrackNet-V.” IEEE Trans. Intell. Transp. Syst. 21 (1): 273–284. https://doi.org/10.1109/TITS.2019.2891167.
Gavilán, M., D. Balcones, O. Marcos, D. F. Llorca, M. A. Sotelo, I. Parra, M. Ocaña, P. Aliseda, P. Yarza, and A. Amírola. 2011. “Adaptive road crack detection system by pavement classification.” Sensors 11 (10): 9628–9657. https://doi.org/10.3390/s111009628.
Huang, J., W. Liu, and X. Sun. 2014. “A pavement crack detection method combining 2D with 3D information based on Dempster-Shafer theory.” Comput.-Aided Civ. Infrastruct. Eng. 29 (4): 299–313. https://doi.org/10.1111/mice.12041.
Huang, Y., and B. Xu. 2006. “Automatic inspection of pavement cracking distress.” J. Electron. Imaging 15 (1): 013017. https://doi.org/10.1117/1.2177650.
Krizhevsky, A., I. Sutskever, and G. E. Hinton. 2017. “ImageNet classification with deep convolutional neural networks.” Commun. ACM 60 (6): 84–90. https://doi.org/10.1145/3065386.
Li, B., K. Wang, A. Zhang, E. Yang, and G. Wang. 2018. “Automatic classification of pavement crack using deep convolutional neural network.” Int. J. Pavement Eng. 21 (4): 1–7. https://doi.org/10.1080/10298436.2018.1485917.
Li, L., W. Ma, L. Li, and C. Lu. 2019. “Research on detection algorithm for bridge cracks based on deep learning.” Acta Autom. Sin. 45 (9): 1727–1742. https://doi.org/10.16383/j.aas.2018.c170052.
Li, Q., Q. Zou, and X. Liu. 2011. “Pavement crack classification via spatial distribution features.” EURASIP J. Adv. Signal Process. 2011 (1): 679675. https://doi.org/10.1155/2011/649675.
Liu, Z., Y. Cao, Y. Wang, and W. Wang. 2019. “Computer vision-based concrete crack detection using U-net fully convolutional networks.” Autom. Constr. 104 (Aug): 129–139. https://doi.org/10.1016/j.autcon.2019.04.005.
Ma, J., X. Zhao, S. He, H. Song, Y. Zhao, H. Song, L. Cheng, J. Wang, Z. Yuan, and F. Huang. 2017. “Review of pavement detection technology.” J. Traffic Transp. Eng 17 (5): 121–137.
Maas, A. L., A. Y. Hannun, and A. Y. Ng. 2013. “Rectifier nonlinearities improve neural network acoustic models.” In Vol. 28 of Proc., Int. Conf. on Machine Learning, 1–6. London: Proceedings of Machine Learning Research.
Mathavan, S., M. Rahman, and K. Kamal. 2015. “Use of a self-organizing map for crack detection in highly textured pavement images.” J. Infrastruct. Syst. 21 (3): 04014052. https://doi.org/10.1061/(ASCE)IS.1943-555X.0000237.
MOT (Ministry of Transport of the People’s Republic of China). 2018. “Development statistics bulletin of transportation industry in 2018.” Accessed April 12, 2019. http://xxgk.mot.gov.cn/jigou/zhghs/201904/t20190412_3186720.html.
Nguyen, T. S., M. Avila, and S. Begot. 2009. “Automatic detection and classification of defect on road pavement using anisotropy measure.” In Proc., European Signal Processing Conf., 617–621. New York: IEEE.
Peng, B., D. Huang, L. Guo, X. Cai, and S. Li. 2018. “Automatic crack detection algorithm from 3D pavement images based on shape analysis at pixel-subpixel level.” J. Chongqing Jiaotong Univ. 37 (9): 34–42. https://doi.org/10.3969/j.issn.1674-0696.2018.09.06.
Peng, B., Y. Jiang, C. Chen, and K. C. P. Wang. 2015. “Automatic parallel cracking detection algorithm based on 1 mm resolution 3D pavement images.” J. Southeast Univ. 45 (6): 1190–1196. https://doi.org/10.3969/j.issn.1001-0505.2015.06.030.
Pratt, H., B. Williams, F. Coenen, and Y. Zheng. 2017.“Fcnn: Fourier convolutional neural networks.” In Proc., Machine Learning and Knowledge Discovery in Databases, 786–798. Dordrecht, Netherlands: Springer.
Ren, S., K. He, R. Girshick, and J. Sun. 2017. “Faster R-CNN: Towards real-time object detection with region proposal networks.” IEEE Trans. Pattern Anal. Mach. Intell. 39 (6): 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031.
Simonyan, K., and A. Zisserman. 2014. “Very deep convolutional networks for large-scale image recognition.” Preprint, submitted September 4, 2014. http://arxiv.org/abs/1409.1556.
Szegedy, C., W. Liu, Y. Jia, P. Sermanet, and A. Rabinovich. 2015.“Going deeper with convolutions.” In Proc., 2015 IEEE Conf. on Computer Vision and Pattern Recognition, 1–12. New York: IEEE. https://doi.org/10.1109/CVPR.2015.7298594.
Zeiler, M. D., and R. Fergus. 2014. “Visualizing and understanding convolutional networks.” In Proc., European Conf. on Computer Vision, 818–833. Dordrecht, Netherlands: Springer.
Zhang, A., K. C. P. Wang, and C. Ai. 2017. “3D shadow modeling for detection of descended patterns on 3D pavement surface.” J. Comput. Civ. Eng. 31 (4): 04017019. https://doi/org/10.1061/(ASCE)CP.1943-5487.0000661.
Zhang, A., K. C. P. Wang, Y. Fei, Y. Liu, S. Tao, C. Chen, J. Q. Li, and B. Li. 2018a. “Deep learning–based fully automated pavement crack detection on 3D asphalt surfaces with an improved CrackNet.” J. Comput. Civ. Eng. 32 (5): 04018041. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000775.
Zhang, K., H. Cheng, and B. Zhang. 2018b. “Unified approach to pavement crack and sealed crack detection using preclassification based on transfer learning.” J. Comput. Civ. Eng. 32 (2): 04018001. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000736.
Zhang, L., F. Yang, Y. D. Zhang, and Y. J. Zhu. 2016. “Road crack detection using deep convolutional neural network.” In Proc., 2016 IEEE Int. Conf. on Image Processing, 3708–3712. New York: IEEE. https://doi.org/10.1109/ICIP.2016.7533052.
Zimmerman, K. A. 2017. “Pavement management systems: Putting data to work.” In Proc., NCHRP Synthesis of Highway Practice. Washington, DC: Transportation Research Board.
Information & Authors
Information
Published In
Journal of Transportation Engineering, Part B: Pavements
Volume 148 • Issue 2 • June 2022
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Feb 4, 2021
Accepted: Jan 12, 2022
Published online: Mar 17, 2022
Published in print: Jun 1, 2022
Discussion open until: Aug 17, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Hong Lang, Jinsong Qian, Ye Yuan, Jiang Chen, Yingying Xing, Aidi Wang, Automatic Pixel-Level Segmentation of Multiple Pavement Distresses and Surface Design Features with PDSNet II, Journal of Computing in Civil Engineering, 10.1061/JCCEE5.CPENG-5894, 38, 6, (2024).
- Tianjie Zhang, Donglei Wang, Yang Lu, Benchmark Study on a Novel Online Dataset for Standard Evaluation of Deep Learning-based Pavement Cracks Classification Models, KSCE Journal of Civil Engineering, 10.1007/s12205-024-1066-8, 28, 4, (1267-1279), (2024).
- Tianjie Zhang, Donglei Wang, Yang Lu, ECSNet: An Accelerated Real-Time Image Segmentation CNN Architecture for Pavement Crack Detection, IEEE Transactions on Intelligent Transportation Systems, 10.1109/TITS.2023.3300312, 24, 12, (15105-15112), (2023).
- Shuo Ding, Hong Lang, Jiang Chen, Ye Yuan, Jian Lu, Automated crack segmentation on 3D asphalt surfaces with richer attention and hybrid pyramid structures, International Journal of Pavement Engineering, 10.1080/10298436.2023.2246097, 24, 1, (2023).
- Luqman Ali, Fady Alnajjar, Wasif Khan, Mohamed Adel Serhani, Hamad Al Jassmi, Bibliometric Analysis and Review of Deep Learning-Based Crack Detection Literature Published between 2010 and 2022, Buildings, 10.3390/buildings12040432, 12, 4, (432), (2022).