Policy Gradient–Based Focal Loss to Reduce False Negative Errors of Convolutional Neural Networks for Pavement Crack Segmentation
Publication: Journal of Infrastructure Systems
Volume 29, Issue 1
Abstract
Convolutional neural networks (CNNs) have achieved tremendous success in pavement crack segmentation. However, it is difficult for CNN-based crack segmentation methods to minimize false-negative and false-positive errors. Compared with false-positive errors, false-negative errors are more difficult to observe and reduce manually. This paper proposes a fine-tuning method for trained CNNs, called policy gradient-based focal loss (focal-PG loss). The trained CNNs will be further trained by focal-PG loss for only one epoch. The proposed focal-PG loss can be applied to reduce the false-negative errors of the trained CNNs by sacrificing their precision. The experimental results show that focal-PG loss greatly improves the crack recognition rate of the trained encoder–decoder network (EDNet). EDNet (focal-PG loss) achieves an overall precision of 96.05%, recall of 99.68%, and F1-score of 97.83% on 100 validation images. In addition, overall precision of 95.53%, recall of 99.58%, and F1-score of 97.51% are observed for the 150 testing images. U-net, LinkNet, and the feature pyramid network are also tested in the paper to validate the effectiveness of focal-PG loss. The results demonstrate that the focal-PG loss can also improve the performance of the aforementioned networks.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request, including the code of the policy gradient-based focal loss.
References
Ashok, A., N. Rhinehart, F. Beainy, and K. M. Kitani. 2017. “N2n learning: Network to network compression via policy gradient reinforcement learning.” Preprint, submitted September 18, 2017. https://arxiv.org/abs/1709.06030.
Ayenu-Prah, A., and N. Attoh-Okine. 2008. “Evaluating pavement cracks with bidimensional empirical mode decomposition.” EURASIP J. Adv. Signal Process. 2008 (Dec): 1–7. https://doi.org/10.1155/2008/861701.
Bray, J., B. Verma, X. Li, and W. He. 2006. “A neural network based technique for automatic classification of road cracks.” In Proc., 2006 IEEE Int. Joint Conf. on Neural Network Proceedings, 907–912. New York: IEEE.
Cao, W., Q. Liu, and Z. He. 2020. “Review of pavement defect detection methods.” IEEE Access 8 (Jan): 14531–14544. https://doi.org/10.1109/ACCESS.2020.2966881.
Chaurasia, A., and E. Culurciello. 2017. “Linknet: Exploiting encoder representations for efficient semantic segmentation.” In Proc., 2017 IEEE Visual Communications and Image Processing (VCIP), 1–4. New York: IEEE.
Chen, Q., Y. Huang, H. Sun, and W. Huang. 2021. “Pavement crack detection using hessian structure propagation.” Adv. Eng. Inf. 49 (Aug): 101303. https://doi.org/10.1016/j.aei.2021.101303.
Chow, J. K., Z. Su, J. Wu, P. S. Tan, X. Mao, and Y.-H. Wang. 2020. “Anomaly detection of defects on concrete structures with the convolutional autoencoder.” Adv. Eng. Inf. 45 (Aug): 101105. https://doi.org/10.1016/j.aei.2020.101105.
Fan, Z., C. Li, Y. Chen, J. Wei, G. Loprencipe, X. Chen, and P. Di Mascio. 2020. “Automatic crack detection on road pavements using encoder-decoder architecture.” Materials (Basel) 13 (13): 2960. https://doi.org/10.3390/ma13132960.
Fan, Z., Y. Wu, J. Lu, and W. Li. 2018. “Automatic pavement crack detection based on structured prediction with the convolutional neural network.” Preprint, submitted February 1, 2018. https://arxiv.org/abs/1802.02208.
Fei, Y., K. C. Wang, A. Zhang, C. Chen, J. Q. Li, Y. Liu, G. Yang, and B. Li. 2019. “Pixel-level cracking detection on 3D asphalt pavement images through deep-learning-based CrackNet-V.” IEEE Trans. Intell. Transp. Syst. 21 (1): 273–284. https://doi.org/10.1109/TITS.2019.2891167.
Gulli, A., and S. Pal. 2017. Deep learning with Keras. Birmingham, UK: Packt Publishing Ltd.
Guo, F., Y. Qian, Y. Wu, Z. Leng, and H. Yu. 2021. “Automatic railroad track components inspection using real-time instance segmentation.” Comput.-Aided Civ. Infrastruct. Eng. 36 (3): 362–377. https://doi.org/10.1111/mice.12625.
He, T., Z. Zhang, H. Zhang, Z. Zhang, J. Xie, and M. Li. 2019. “Bag of tricks for image classification with convolutional neural networks.” In Proc., IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 558–567. New York: IEEE.
Hu, W., W. Wang, C. Ai, J. Wang, W. Wang, X. Meng, J. Liu, H. Tao, and S. Qiu. 2021. “Machine vision-based surface crack analysis for transportation infrastructure.” Autom. Constr. 132 (Dec): 103973. https://doi.org/10.1016/j.autcon.2021.103973.
Huyan, J., W. Li, S. Tighe, Z. Xu, and J. Zhai. 2020. “CrackU-net: A novel deep convolutional neural network for pixelwise pavement crack detection.” Struct. Control Health Monit. 27 (8): e2551. https://doi.org/10.1002/stc.2551.
Huyan, J., W. Li, S. Tighe, J. Zhai, Z. Xu, and Y. Chen. 2019. “Detection of sealed and unsealed cracks with complex backgrounds using deep convolutional neural network.” Autom. Constr. 107 (Nov): 102946. https://doi.org/10.1016/j.autcon.2019.102946.
Kingma, D. P., and J. Ba. 2014. “Adam: A method for stochastic optimization.” Preprint, submitted December 22, 2014. https://arxiv.org/abs/1412.6980.
Liao, P.-S., T.-S. Chen, and P.-C. Chung. 2001. “A fast algorithm for multilevel thresholding.” J. Inf. Sci. Eng. 17 (5): 713–727.
Lin, T.-Y., P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie. 2017a. “Feature pyramid networks for object detection.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 2117–2125. New York: IEEE.
Lin, T.-Y., P. Goyal, R. Girshick, K. He, and P. Dollár. 2017b. “Focal loss for dense object detection.” In Proc., IEEE Int. Conf. on Computer Vision, 2980–2988. New York: IEEE.
Liu, J., X. Yang, S. Lau, X. Wang, S. Luo, V. C.-S. Lee, and L. Ding. 2020. “Automated pavement crack detection and segmentation based on two-step convolutional neural network.” Comput.-Aided Civ. Infrastruct. Eng. 35 (11): 1291–1305. https://doi.org/10.1111/mice.12622.
Liu, Z., Y. Cao, Y. Wang, and W. Wang. 2019. “Computer vision-based concrete crack detection using U-net fully convolutional networks.” Autom. Constr. 104 (Aug): 129–139. https://doi.org/10.1016/j.autcon.2019.04.005.
Maode, Y., B. Shaobo, X. Kun, and H. Yuyao. 2007. “Pavement crack detection and analysis for high-grade highway.” In Proc., 2007 8th Int. Conf. on Electronic Measurement and Instruments, 548–552. New York: IEEE.
Mei, Q., M. Gül, and M. R. Azim. 2020. “Densely connected deep neural network considering connectivity of pixels for automatic crack detection.” Autom. Constr. 110 (Feb): 103018. https://doi.org/10.1016/j.autcon.2019.103018.
Merazi-Meksen, T., M. Boudraa, and B. Boudraa. 2014. “Mathematical morphology for TOFD image analysis and automatic crack detection.” Ultrasonics 54 (6): 1642–1648. https://doi.org/10.1016/j.ultras.2014.03.005.
Nhat-Duc, H., Q.-L. Nguyen, and V.-D. Tran. 2018. “Automatic recognition of asphalt pavement cracks using metaheuristic optimized edge detection algorithms and convolution neural network.” Autom. Constr. 94 (Oct): 203–213. https://doi.org/10.1016/j.autcon.2018.07.008.
Nie, M., and K. Wang. 2018. “Pavement distress detection based on transfer learning.” In Proc., 2018 5th Int. Conf. on Systems and Informatics (ICSAI), 435–439. New York: IEEE.
Nisanth, A., and A. Mathew. 2014. “Automated visual inspection on pavement crack detection and characterization.” Int. J. Technol. Eng. Syst. 6 (1): 14–20.
Otsu, N. 1979. “A threshold selection method from gray-level histograms.” IEEE Trans. Syst. Man Cybern. 9 (1): 62–66. https://doi.org/10.1109/TSMC.1979.4310076.
Polovnikov, V., D. Alekseev, I. Vinogradov, and G. V. Lashkia. 2021. “DAUNet: Deep augmented neural network for pavement crack segmentation.” IEEE Access 9 (Sep): 125714–125723. https://doi.org/10.1109/ACCESS.2021.3111223.
Redmon, J., and A. Farhadi. 2018. “Yolov3: An incremental improvement.” Preprint, submitted April 8, 2018. https://arxiv.org/abs/1804.02767.
Ren, Y., J. Huang, Z. Hong, W. Lu, J. Yin, L. Zou, and X. Shen. 2020. “Image-based concrete crack detection in tunnels using deep fully convolutional networks.” Constr. Build. Mater. 234 (Feb): 117367. https://doi.org/10.1016/j.conbuildmat.2019.117367.
Ronneberger, O., P. Fischer, and T. Brox. 2015. “U-net: Convolutional networks for biomedical image segmentation.” In Proc., Int. Conf. on Medical Image Computing and Computerassisted Intervention, 234–241. Dordrecht, Netherlands: Springer.
Tang, Y., A. A. Zhang, L. Luo, G. Wang, and E. Yang. 2021. “Pixel-level pavement crack segmentation with encoder-decoder network.” Measurement 184 (Nov): 109914. https://doi.org/10.1016/j.measurement.2021.109914.
Tsai, D.-M., and P.-H. Jen. 2021. “Autoencoder-based anomaly detection for surface defect inspection.” Adv. Eng. Inf. 48 (Apr): 101272. https://doi.org/10.1016/j.aei.2021.101272.
Wan, H., L. Gao, M. Su, Q. Sun, and L. Huang. 2021. “Attention-based convolutional neural network for pavement crack detection.” Adv. Mater. Sci. Eng. 2021 (Apr): 5520515. https://doi.org/10.1155/2021/5520515.
Wang, W., M. Wang, H. Li, H. Zhao, K. Wang, C. He, J. Wang, S. Zheng, and J. Chen. 2019. “Pavement crack image acquisition methods and crack extraction algorithms: A review.” J. Traffic Transp. Eng. 6 (6): 535–556. https://doi.org/10.1016/j.jtte.2019.10.001.
Wang, W., A. Zhang, K. C. Wang, A. F. Braham, and S. Qiu. 2018. “Pavement crack width measurement based on Laplace’s equation for continuity and unambiguity.” Comput.-Aided Civ. Infrastruct. Eng. 33 (2): 110–123. https://doi.org/10.1111/mice.12319.
Wang, Y., K. Song, J. Liu, H. Dong, Y. Yan, and P. Jiang. 2021. “RENet: Rectangular convolution pyramid and edge enhancement network for salient object detection of pavement cracks.” Measurement 170 (Jan): 108698. https://doi.org/10.1016/j.measurement.2020.108698.
Xu, Z., H. Guan, J. Kang, X. Lei, L. Ma, Y. Yu, Y. Chen, and J. Li. 2022. “Pavement crack detection from CCD images with a locally enhanced transformer network.” Int. J. Appl. Earth Obs. Geoinf. 110 (Jun): 102825. https://doi.org/10.1016/j.jag.2022.102825.
Zhang, A., K. C. Wang, Y. Fei, Y. Liu, C. Chen, G. Yang, J. Q. Li, E. Yang, and S. Qiu. 2019. “Automated pixel-level pavement crack detection on 3D asphalt surfaces with a recurrent neural network.” Comput.-Aided Civ. Infrastruct. Eng. 34 (3): 213–229. https://doi.org/10.1111/mice.12409.
Zhang, A., K. C. Wang, Y. Fei, Y. Liu, S. Tao, C. Chen, J. Q. Li, and B. Li. 2018. “Deep learning–based fully automated pavement crack detection on 3D asphalt surfaces with an improved CrackNet.” J. Comput. Civ. Eng. 32 (5): 04018041. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000775.
Zhang, A., K. C. Wang, B. Li, E. Yang, X. Dai, Y. Peng, Y. Fei, Y. Liu, J. Q. Li, and C. Chen. 2017a. “Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network.” Comput.-Aided Civ. Infrastruct. Eng. 32 (10): 805–819. https://doi.org/10.1111/mice.12297.
Zhang, D., Q. Li, Y. Chen, M. Cao, L. He, and B. Zhang. 2017b. “An efficient and reliable coarse-to-fine approach for asphalt pavement crack detection.” Image Vision Comput. 57 (Jan): 130–146. https://doi.org/10.1016/j.imavis.2016.11.018.
Zou, Q., Z. Zhang, Q. Li, X. Qi, Q. Wang, and S. Wang. 2018. “DeepCrack: Learning hierarchical convolutional features for crack detection.” IEEE Trans. Image Process. 28 (3): 1498–1512. https://doi.org/10.1109/TIP.2018.2878966.
Information & Authors
Information
Published In
Journal of Infrastructure Systems
Volume 29 • Issue 1 • March 2023
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Mar 7, 2022
Accepted: Oct 26, 2022
Published online: Jan 6, 2023
Published in print: Mar 1, 2023
Discussion open until: Jun 6, 2023
ASCE Technical Topics:
- Artificial intelligence and machine learning
- Computer programming
- Computing in civil engineering
- Continuum mechanics
- Cracking
- Engineering fundamentals
- Engineering mechanics
- Fracture mechanics
- Gravels
- Infrastructure
- Methodology (by type)
- Neural networks
- Pavement condition
- Pavements
- Research methods (by type)
- Solid mechanics
- Transportation engineering
- Validation
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.