Crack Detection via Hierarchical Multiscale Feature Learning and Densely Connected Conditional Random Field
Publication: ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems, Part A: Civil Engineering
Volume 10, Issue 1
Abstract
Crack analysis based on computer vision has become a common approach for crack detection and localization in civil infrastructure. In practice, many cracks show poor continuity, uneven gray levels, low contrast, complex topology, and background noise. These characteristics present significant difficulties for image-based crack detection. In this paper, we propose a novel framework that includes a deep fully convolutional network and densely connected conditional random field (dense CRF) to realize pixel-level crack detection in an end-to-end manner. The network learns and aggregates multilevel features at hierarchical convolutional stages. Specifically, the backbone of our network is novel self-attention modules with convolution kernels for context information extraction across channels, and the network end with multiple parallel atrous convolution filters with different rate to capture objects and features at multiple scales. Finally, we combine the network output with a dense CRF to refine the final prediction results. The network in our study is trained and evaluated using three classical crack data sets. The experimental results clearly demonstrate that our method outperforms other approaches in terms of performance.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This work is supported by the Natural Science Foundation of Hunan Province under Grant 2022JJ30012.
References
Cha, Y.-J., W. Choi, and O. Büyüköztürk. 2017. “Deep learning-based crack damage detection using convolutional neural networks.” Comput.-Aided Civ. Infrastruct. Eng. 32 (5): 361–378. https://doi.org/10.1111/mice.12263.
Chen, L.-C., G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. 2017. “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs.” IEEE Trans. Pattern Anal. Mach. Intell. 40 (4): 834–848. https://doi.org/10.1109/TPAMI.2017.2699184.
Cogswell, M., X. Lin, S. Purushwalkam, and D. Batra. 2014. “Combining the best of graphical models and convnets for semantic segmentation.” Preprint, submitted December 14, 2014. http://arxiv.org/abs/1412.4313.
Das, S., S. Dutta, C. Putcha, S. Majumdar, and D. Adak. 2020. “A data-driven physics-informed method for prognosis of infrastructure systems: Theory and application to crack prediction.” ASCE-ASME J. Risk Uncertainty Eng. Syst. Part A: Civ. Eng. 6 (2): 04020013. https://doi.org/10.1061/AJRUA6.0001053.
Deng, J., A. Singh, Y. Zhou, Y. Lu, and V. C.-S. Lee. 2022. “Review on computer vision-based crack detection and quantification methodologies for civil structures.” Constr. Build. Mater. 356 (May): 129238. https://doi.org/10.1016/j.conbuildmat.2022.129238.
Dias-da Costa, D., J. Valença, E. Júlio, and H. Araújo. 2017. “Crack propagation monitoring using an image deformation approach.” Struct. Control Health Monit. 24 (10): e1973. https://doi.org/10.1002/stc.1973.
Eigen, D., and R. Fergus. 2015. “Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture.” In Proc., IEEE Int. Conf. on Computer Vision, 2650–2658. New York: IEEE.
Fei, Y., K. C. Wang, A. Zhang, C. Chen, J. Q. Li, Y. Liu, G. Yang, and B. Li. 2019. “Pixel-level cracking detection on 3d asphalt pavement images through deep-learning-based cracknet-v.” IEEE Trans. Intell. Transp. Syst. 21 (1): 273–284. https://doi.org/10.1109/TITS.2019.2891167.
Gao, Y., P. Zhai, and K. M. Mosalam. 2021. “Balanced semisupervised generative adversarial network for damage assessment from low-data imbalanced-class regime.” Comput.-Aided Civ. Infrastruct. Eng. 36 (9): 1094–1113. https://doi.org/10.1111/mice.12741.
Gegúndez-Arias, M. E., A. Aquino, J. M. Bravo, and D. Marín. 2011. “A function for quality evaluation of retinal vessel segmentations.” IEEE Trans. Med. Imaging 31 (2): 231–239. https://doi.org/10.1109/TMI.2011.2167982.
Girshick, R. 2015. “Fast r-cnn.” In Proc., IEEE Int. Conf. on Computer Vision, 1440–1448. New York: IEEE.
Hsieh, Y.-A., and Y. J. Tsai. 2020. “Machine learning for crack detection: Review and model performance comparison.” J. Comput. Civ. Eng. 34 (5): 04020038. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000918.
Hu, H., J. Gu, Z. Zhang, J. Dai, and Y. Wei. 2018. “Relation networks for object detection.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 3588–3597. New York: IEEE.
Krähenbühl, P., and V. Koltun. 2011. “Efficient inference in fully connected crfs with gaussian edge potentials.” Adv. Neural Inf. Process. Syst. 24: 109–117. https://doi.org/10.48550/arXiv.1210.5644.
Krizhevsky, A., I. Sutskever, and G. E. Hinton. 2017. “Imagenet classification with deep convolutional neural networks.” Commun. ACM 60 (6): 84–90. https://doi.org/10.1145/3065386.
Liang, X. 2019. “Image-based post-disaster inspection of reinforced concrete bridge systems using deep learning with bayesian optimization.” Comput.-Aided Civ. Infrastruct. Eng. 34 (5): 415–430. https://doi.org/10.1111/mice.12425.
Liu, H., X. Miao, C. Mertz, C. Xu, and H. Kong. 2021. “Crackformer: Transformer network for fine-grained crack detection.” In Proc., IEEE/CVF Int. Conf. on Computer Vision, 3783–3792. New York: IEEE.
Liu, Y., M.-M. Cheng, X. Hu, K. Wang, and X. Bai. 2017. “Richer convolutional features for edge detection.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 3000–3009. New York: IEEE.
Liu, Y., J. Yao, X. Lu, R. Xie, and L. Li. 2019. “Deepcrack: A deep hierarchical feature learning architecture for crack segmentation.” Neurocomputing 338 (Aug): 139–153. https://doi.org/10.1016/j.neucom.2019.01.036.
Long, J., E. Shelhamer, and T. Darrell. 2015. “Fully convolutional networks for semantic segmentation.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 3431–3440. New York: IEEE.
Maeda, H., T. Kashiyama, Y. Sekimoto, T. Seto, and H. Omata. 2021. “Generative adversarial network for road damage detection.” Comput.-Aided Civ. Infrastruct. Eng. 36 (1): 47–60. https://doi.org/10.1111/mice.12561.
Robinson, E. I., J. Marzat, and T. Raïssi. 2018. “Filtering and uncertainty propagation methods for model-based prognosis of fatigue crack growth in unidirectional fiber-reinforced composites.” ASCE-ASME J. Risk Uncertainty Eng. Syst. Part A: Civ. Eng. 4 (4): 04018040. https://doi.org/10.1061/AJRUA6.0000991.
Spencer, B. F., Jr., V. Hoskere, and Y. Narazaki. 2019. “Advances in computer vision-based civil infrastructure inspection and monitoring.” Engineering 5 (2): 199–222. https://doi.org/10.1016/j.eng.2018.11.030.
Wang, F., M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, and X. Tang. 2017. “Residual attention network for image classification.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 3156–3164. New York: IEEE.
Xie, S., and Z. Tu. 2015. “Holistically-nested edge detection.” In Proc., IEEE Int. Conf. on Computer Vision, 1395–1403. New York: IEEE.
Xu, T., S. Ding, H. Zhou, and G. Li. 2023. “Establishment of the off-center embedded crack stress intensity factor database for probabilistic risk assessment based on universal weight function.” ASCE-ASME J. Risk Uncertainty Eng. Syst. Part B: Mech. Eng. 9 (2): 021202. https://doi.org/10.1115/1.4055535.
Zhang, A., K. C. Wang, B. Li, E. Yang, X. Dai, Y. Peng, Y. Fei, Y. Liu, J. Q. Li, and C. Chen. 2017. “Automated pixel-level pavement crack detection on 3d asphalt surfaces using a deep-learning network.” Comput.-Aided Civ. Infrastruct. Eng. 32 (10): 805–819. https://doi.org/10.1111/mice.12297.
Zhang, Z., X. Liu, Y. Zhang, M. Zhou, and J. Chen. 2020. “Time interval of multiple crossings of the wiener process and a fixed threshold in engineering.” Mech. Syst. Signal Process. 135 (Feb): 106389. https://doi.org/10.1016/j.ymssp.2019.106389.
Zhang, Z., C. Zhao, Z. Zhao, F. Wang, and B. Zhao. 2023. “Structural fatigue reliability evaluation based on probability analysis of the number of zero-crossings of stochastic response process.” Eng. Fail. Anal. 143 (Jan): 106923. https://doi.org/10.1016/j.engfailanal.2022.106923.
Zhang, Z., M. Zhou, and M. Fang. 2021. “First-passage probability analysis of wiener process using different methods and its applications in the evaluation of structural durability degradation.” Eur. J. Environ. Civ. Eng. 25 (10): 1763–1781. https://doi.org/10.1080/19648189.2019.1601134.
Zheng, Y., Y. Gao, S. Lu, and K. M. Mosalam. 2022. “Multistage semisupervised active learning framework for crack identification, segmentation, and measurement of bridges.” Comput.-Aided Civ. Infrastruct. Eng. 37 (9): 1089–1108. https://doi.org/10.1111/mice.12851.
Zou, Q., Z. Zhang, Q. Li, X. Qi, Q. Wang, and S. Wang. 2018. “Deepcrack: Learning hierarchical convolutional features for crack detection.” IEEE Trans. Image Process. 28 (3): 1498–1512. https://doi.org/10.1109/TIP.2018.2878966.
Information & Authors
Information
Published In
ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems, Part A: Civil Engineering
Volume 10 • Issue 1 • March 2024
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Mar 31, 2023
Accepted: Aug 26, 2023
Published online: Oct 20, 2023
Published in print: Mar 1, 2024
Discussion open until: Mar 20, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.