Preprocessing of Crack Recognition: Automatic Crack-Location Method Based on Deep Learning
Publication: Journal of Materials in Civil Engineering
Volume 35, Issue 3
Abstract
With advancements in artificial intelligence and computer vision, machine learning has become widely employed in location and detection of road pavement distresses. Recently, recognition methods based on convolutional neural networks (CNNs) have been implemented to segment pavement cracks at pixel level in order to evaluate the pavement condition. However, this method usually consists of some common processes, including manually predetermining the approximate location of cracks followed by selecting the image containing the cracks and then performing pixel-level segmentation, which is why it is worth automating the preprocessing to replace the manual selection step. Moreover, the issues of a low proportion of positive samples, complex crack topologies, different inset conditions, and complex pavement background make the task of automatic pavement location more challenging. Therefore, this paper proposes a novel method for preprocessing crack recognition, which automatically locates cracks and yields great savings in labor costs. Specifically, a real-world road pavement crack data set obtained from a common digital camera mounted on a vehicle is built to test the proposed crack location method, called Double-Head. It improves the accuracy of crack object localization by using an independent fully connected head (fc-head) and a convolution head (conv-head). The results show that our method improves average precision (AP) 6.5% over Faster R-CNN using only a fc-head, and outperforms many advanced object detection methods.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code generated or used during the study are proprietary or confidential in nature and may only be provided with restrictions. Information about the collection area are kept confidential for national security reasons. Also, the raw/processed data required to reproduce these findings cannot be shared at this time as the data forms part of an ongoing study.
Acknowledgments
This research was funded by the Natural Science Foundation of Jiangsu Province under Grant BK20210720, the Natural Science Foundation for Colleges and Universities in Jiangsu Province under Grant 19KJB580018, and the Housing and Urban Rural Development System Scientific Project of Jiangsu Province under Grant 2021ZD02.
References
Cha, Y.-J., W. Choi, and O. Büyüköztürk. 2017. “Deep learning-based crack damage detection using convolutional neural networks: Deep learning-based crack damage detection using CNNs.” Comput.-Aided Civ. Infrastruct. Eng. 32 (5): 361–378. https://doi.org/10.1111/mice.12263.
Chambon, S., P. Subirats, and J. Dumoulin. 2009. “Introduction of a wavelet transform based on 2D matched filter in a Markov random field for fine structure extraction: Application on road crack detection.” In Vol. 7251 of Image processing: Machine vision applications II, edited by K. S. Niel and D. Fofi. San Jose, CA: SPIE.
Chen, F.-C., and M. R. Jahanshahi. 2018. “NB-CNN: Deep learning-based crack detection using convolutional neural network and naïve Bayes data fusion.” IEEE Trans. Ind. Electron. 65 (5): 4392–4400. https://doi.org/10.1109/TIE.2017.2764844.
Cubero-Fernandez, A., F. Rodriguez-Lozano, R. Villatoro, J. Olivares, and J. M. Palomares. 2017. “Efficient pavement crack detection and classification.” EURASIP J. Image Video Process. 2017 (1): 1–11. https://doi.org/10.1186/s13640-017-0187-0.
Dorafshan, S., R. J. Thomas, and M. Maguire. 2018. “Comparison of deep convolutional neural networks and edge detectors for image-based crack detection in concrete.” Constr. Build. Mater. 186 (Oct): 1031–1045. https://doi.org/10.1016/j.conbuildmat.2018.08.011.
Girshick, R. 2015. “Fast R-CNN.” Preprint, submitted April 30, 2015. http://arxiv.org/abs/1504.08083.
Gotmare, A., N. Shirish Keskar, C. Xiong, and R. Socher. 2018. “A closer look at deep learning heuristics: Learning rate restarts, warmup and distillation.” Preprint, submitted October 29, 2018. http://arxiv.org/abs/1810.13243.
He, K., G. Gkioxari, P. Dollár, and R. Girshick. 2017. “Mask R-CNN.” In Proc., IEEE Int. Conf. on Computer Vision, 2961–2969. New York: IEEE.
He, K., X. Zhang, S. Ren, and J. Sun. 2015. “Deep residual learning for image recognition.” Preprint, submitted December 10, 2015. http://arxiv.org/abs/1512.03385.
Heideklang, R., and P. Shokouhi. 2015. “Multi-sensor image fusion at signal level for improved near-surface crack detection.” NDT & E Int. 71 (Apr): 16–22. https://doi.org/10.1016/j.ndteint.2014.12.008.
Hou, Y., Q. Li, C. Zhang, G. Lu, Z. Ye, Y. Chen, L. Wang, and D. Cao. 2021. “The state-of-the-art review on applications of intrusive sensing, image processing techniques, and machine learning methods in pavement monitoring and analysis.” Engineering 7 (6): 845–856. https://doi.org/10.1016/j.eng.2020.07.030.
Hou, Y., S. Liu, D. Cao, B. Peng, Z. Liu, W. Sun, and N. Chen. 2022. “A deep learning method for pavement crack identification based on limited field images.” IEEE Trans. Intell. Transp. Syst. (Mar): 1–10. https://doi.org/10.1109/TITS.2022.3160524.
Krizhevsky, A., I. Sutskever, and G. E. Hinton. 2017. “ImageNet classification with deep convolutional neural networks.” Commun. ACM 60 (6): 84–90. https://doi.org/10.1145/3065386.
Landström, A., and M. J. Thurley. 2012. “Morphology-based crack detection for steel slabs.” IEEE J. Sel. Top. Signal Process. 6 (7): 866–875. https://doi.org/10.1109/JSTSP.2012.2212416.
Lee, D., J. Kim, and D. Lee. 2019. “Robust concrete crack detection using deep learning-based semantic segmentation.” Int. J. Aeronaut. Space Sci. 20 (1): 287–299. https://doi.org/10.1007/s42405-018-0120-5.
Li, B., K. C. P. Wang, A. Zhang, E. Yang, and G. Wang. 2020. “Automatic classification of pavement crack using deep convolutional neural network.” Int. J. Pavement Eng. 21 (4): 457–463. https://doi.org/10.1080/10298436.2018.1485917.
Li, Q., Q. Zou, D. Zhang, and Q. Mao. 2011. “FoSA: F* seed-growing approach for crack-line detection from pavement images.” Image Vision Comput. 29 (12): 861–872. https://doi.org/10.1016/j.imavis.2011.10.003.
Lin, T. Y., P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie. 2017. “Feature pyramid networks for object detection.” Preprint, submitted December 9, 2016. http://arxiv.org/abs/1612.03144.
Lin, T. Y., P. Goyal, R. Girshick, K. He, and P. Dollár. 2018. “Focal loss for dense object detection.” Preprint, submitted August 7, 2017. http://arxiv.org/abs/1708.02002.
Lin, T. Y., M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. 2014. “Microsoft COCO: Common objects in context.” In Vol. 8693 of Computer vision: ECCV 2014: Lecture notes in computer science, edited by D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, 740–755. Cham, Switzerland: Springer.
Long, J., E. Shelhamer, and T. Darrell. 2015. “Fully convolutional networks for semantic segmentation.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition. New York: IEEE.
Maeda, H., Y. Sekimoto, T. Seto, T. Kashiyama, and H. Omata. 2018. “Road damage detection and classification using deep neural networks with smartphone images: Road damage detection and classification.” Comput.-Aided Civ. Infrastruct. Eng. 33 (12): 1127–1141. https://doi.org/10.1111/mice.12387.
Mandal, V., L. Uong, and Y. Adu-Gyamfi. 2018. “Automated road crack detection using deep convolutional neural networks.” In Proc., 2018 IEEE Int. Conf. on Big Data (Big Data), 5212–5215. New York: IEEE. https://doi.org/10.1109/BigData.2018.8622327.
Miao, P., and T. Srimahachota. 2021. “Cost-effective system for detection and quantification of concrete surface cracks by combination of convolutional neural network and image processing techniques.” Constr. Build. Mater. 293 (Jul): 123549. https://doi.org/10.1016/j.conbuildmat.2021.123549.
Nguyen, T. S., S. Begot, F. Duculty, and M. Avila. 2011. “Free-form anisotropy: A new method for crack detection on pavement surface images.” In Proc., 2011 18th IEEE Int. Conf. on Image Processing, 1069–1072. Brussels: IEEE. https://doi.org/10.1109/ICIP.2011.6115610.
Nishikawa, T., J. Yoshida, T. Sugiyama, and Y. Fujino. 2012. “Concrete crack detection by multiple sequential image filtering: Concrete crack detection by image processing.” Comput.-Aided Civ. Infrastruct. Eng. 27 (1): 29–47. https://doi.org/10.1111/j.1467-8667.2011.00716.x.
Redmon, J., and A. Farhadi. 2018. “YOLO v3: An incremental improvement.” Preprint, submitted April 8, 2018. http://arxiv.org/abs/1804.02767.
Ren, S., K. He, R. Girshick, and J. Sun. 2015. “Faster R-CNN: Towards real-time object detection with region proposal networks.” Preprint, submitted June 4, 2016. http://arxiv.org/abs/1506.01497.
Ren, S., K. He, R. Girshick, and J. Sun. 2016. “Faster R-CNN: Towards real-time object detection with region proposal networks.” Preprint, submitted June 4, 2015. http://arxiv.org/abs/1506.01497.
Ren, Y., J. Huang, Z. Hong, W. Lu, J. Yin, L. Zou, and X. Shen. 2020. “Image-based concrete crack detection in tunnels using deep fully convolutional networks.” Constr. Build. Mater. 234 (Feb): 117367. https://doi.org/10.1016/j.conbuildmat.2019.117367.
Ronneberger, O., P. Fischer, and T. Brox. 2015. “U-Net: Convolutional networks for biomedical image segmentation.” In Vol. 9351 of Medical image computing and computer-assisted intervention: MICCAI 2015: Lecture notes in computer science, edited by N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi, 234–241. Cham, Switzerland: Springer International Publishing.
Salman, M., S. Mathavan, K. Kamal, and M. Rahman. 2013. “Pavement crack detection using the Gabor filter.” In Proc., 16th Int. IEEE Conf. on Intelligent Transportation Systems (ITSC 2013), 2039–2044. The Hague, Netherlands: IEEE. https://doi.org/10.1109/ITSC.2013.6728529.
Talab, A. M. A., Z. Huang, F. Xi, and L. HaiMing. 2016. “Detection crack in image using Otsu method and multiple filtering in image processing techniques.” Optik 127 (3): 1030–1033. https://doi.org/10.1016/j.ijleo.2015.09.147.
Tian, Z., C. Shen, H. Chen, and T. He. 2019. “FCOS: Fully convolutional one-stage object detection.” In Proc., 2019 IEEE/CVF Int. Conf. on Computer Vision (ICCV), 9626–9635. New York: IEEE. https://doi.org/10.1109/ICCV.2019.00972.
Tong, Z., J. Gao, Z. Han, and Z. Wang. 2018. “Recognition of asphalt pavement crack length using deep convolutional neural networks.” Road Mater. Pavement Des. 19 (6): 1334–1349. https://doi.org/10.1080/14680629.2017.1308265.
Wang, K. C., and W. Gong. 2005. “Real-time automated survey system of pavement cracking in parallel environment.” J. Infrastruct. Syst. 11 (3): 154–164. https://doi.org/10.1061/(ASCE)1076-0342(2005)11:3(154).
Wang, W. X., W. S. Li, and X. Yu. 2012. “Fractional differential algorithms for rock fracture images.” Imag Sci. J. 60 (2): 103–111. https://doi.org/10.1179/1743131X11Y.0000000012.
Wu, Y., Y. Chen, L. Yuan, Z. Liu, L. Wang, H. Li, and Y. Fu. 2020. “Rethinking classification and localization for object detection.” In Proc., IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 10186–10195. New York: IEEE. https://doi.org/10.1109/CVPR42600.2020.01020.
Xu, Z., Y. Che, H. Min, Z. Wang, and X. Zhao. 2019. “Initial classification algorithm for pavement distress images using features fusion.” In Vol. 98 of Intelligent interactive multimedia systems and services: Smart innovation, systems and technologies, edited by G. De Pietro, L. Gallo, R. J. Howlett, L. C. Jain, and L. Vlacic, 418–427. Cham, Switzerland: Springer International Publishing.
Yang, Z., S. Liu, H. Hu, L. Wang, and S. Lin. 2019. “RepPoints: Point set representation for object detection.” In Proc., 2019 IEEE/CVF Int. Conf. on Computer Vision (ICCV), 9656–9665. New York: IEEE. https://doi.org/10.1109/ICCV.2019.00975.
Zhang, A., Q. Li, K. C. Wang, and S. Qiu. 2013. “Matched filtering algorithm for pavement cracking detection.” Transp. Res. Rec. 2367 (1): 30–42. https://doi.org/10.3141/2367-04.
Zhang, L., F. Yang, Y. D. Zhang, and Y. J. Zhu. 2016. “Road crack detection using deep convolutional neural network.” In Proc., 2016 IEEE Int. Conf. on Image Processing (ICIP), 3708–3712. New York: IEEE. https://doi.org/10.1109/ICIP.2016.7533052.
Zhu, C., Y. He, and M. Savvides. 2019. “Feature selective anchor-free module for single-shot object detection.” In Proc., 2019 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 840–849. New York: IEEE. https://doi.org/10.1109/CVPR.2019.00093.
Information & Authors
Information
Published In
Journal of Materials in Civil Engineering
Volume 35 • Issue 3 • March 2023
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Apr 21, 2022
Accepted: May 27, 2022
Published online: Dec 21, 2022
Published in print: Mar 1, 2023
Discussion open until: May 21, 2023
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.