EVPTMFF: Bridge Crack Detection Based on Efficient Visual Pyramid Transformer and Multiple-Feature Fusion
Publication: Journal of Performance of Constructed Facilities
Volume 38, Issue 4
Abstract
One of the key tasks to ensure infrastructure safety is the periodic detection of bridge cracks. Since manual crack detection is subjective and inefficient, it is very important to develop an automatic crack recognition system by using machine vision. Inspired by the pyramid vision transformer (PVT) and the feature pyramid network (FPN) variants, a method combining PVT, residual transformer (REST), holistically nested edge detection (HED), and downstream detection tasks is proposed in this paper, which is named EVPTMFF (efficient visual pyramid transformer and multiple-feature fusion). Based on the PVT, the multiheaded attention module was replaced and the efficient attention module was adopted, which could process the data efficiently and flexibly. To improve the performance of EVPTMFF, the original perceptual field windows were changed. The adjacent windows were partially overlapped, which was more conducive to feature interaction and improves detection performance. To prove the generalization ability of the model, three different data sets related to bridges were collected and formed. We carried out experiments on these three data sets, and EVPTMFF showed good results. Especially for larger data sets, the performance advantage was more significant.
Practical Applications
The crack detection model proposed in this paper presents a good detection effect under different illumination and interference. And the detection results, collected data, time, and other information are saved to the bridge crack detection software. This can help engineers quickly and accurately detect cracks on the bridge surface, as well as predict the development trend of cracks and possible safety issues. In practical application, the bridge crack detection system can help engineers find and solve the bridge crack problem in time and avoid the security risks and economic losses caused by cracks. At the same time, the efficiency and accuracy of bridge maintenance can be improved, and the maintenance cost and time can be reduced. The bridge crack detection system can be integrated with other hardware equipment and management systems to form a complete bridge management platform, which contributes to the traffic construction and social and economic development of the city.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code supporting the findings of this study may be obtained from the corresponding author upon reasonable request.
Acknowledgments
The research is jointly supported by the Key Research and Development Program of Shaanxi (2023-YBGY-264,2020ZDLGY09-03) and the Key Research and Development Program of Guangxi (GK-AB20159032).
References
Abdel-Qader, I., O. Abudayyeh, and M. E. Kelly. 2003. “Analysis of edge-detection techniques for crack identification in bridges.” J. Comput. Civ. Eng. 17 (4): 255–263. https://doi.org/10.1061/(ASCE)0887-3801(2003)17:4(255).
Ali, R., J. H. Chuah, M. S. A. Talip, N. Mokhtar, and M. A. Shoaib. 2022. “Structural crack detection using deep convolutional neural networks.” Autom. Constr. 133 (Jan): 103989.
Chen, Q., Y. Huang, X. Weng, and W. Liu. 2021. “Curve-based crack detection using crack information gain.” Struct. Control Health Monit. 28 (8): e2764. https://doi.org/10.1002/stc.2764.
Choi, S., K. Kim, J. Lee, S. H. Park, H.-J. Lee, and J. Yoon. 2019. “Image processing algorithm for real-time crack inspection in hole expansion test.” Int. J. Precis. Eng. Manuf. 20 (7): 1139–1148. https://doi.org/10.1007/s12541-019-00101-4.
Dais, D., İ. E. Bal, E. Smyrou, and V. Sarhosis. 2021. “Automatic crack classification and segmentation on masonry surfaces using convolutional neural networks and transfer learning.” Autom. Constr. 125 (May): 103606.
Dosovitskiy, A., L. Beyer, A. Kolesnikov, D. Weissenborn, and N. Houlsby. 2020. “An image is worth 16x16 words: Transformers for image recognition at scale.” Preprint, submitted October 22, 2020. https://arxiv.org/abs/2010.11929.
Dung, C. V., and L. D. Anh. 2019. “Autonomous concrete crack detection using deep fully convolutional neural network.” Autom. Constr. 99 (Mar): 52–58. https://doi.org/10.1016/j.autcon.2018.11.028.
Dung, C. V., H. Sekiya, S. Hirano, T. Okatani, and C. Miki. 2019. “A vision-based method for crack detection in gusset plate welded joints of steel bridges using deep convolutional neural networks.” Autom. Constr. 102 (Jun): 217–229. https://doi.org/10.1016/j.autcon.2019.02.013.
Fang, F., L. Li, Y. Gu, H. Zhu, and J.-H. Lim. 2020. “A novel hybrid approach for crack detection.” Pattern Recognit. 107 (Nov): 107474.
Fu, X., N. Cai, K. Huang, H. Wang, P. Wang, C. Liu, and H. Wang. 2019. “M-Net: A novel U-net with multi-stream feature fusion and multi-scale dilated convolutions for bile ducts and hepatolith segmentation.” IEEE Access 7 (Oct): 148645–148657. https://doi.org/10.1109/ACCESS.2019.2946582.
Han, R., X. Wang, N. Bai, Q. Wang, Z. Liu, and J. Xue. 2023. “FCD-Net: Learning to detect multiple types of homologous deepfake face images.” IEEE Trans. Inf. Forensics Secur. 18: 2653–2666.
Huang, X., X. Zhang, Y. Xiong, and Y. Zhang. 2023. “Intelligent fault diagnosis of turbine blade cracks via multiscale sparse filtering and multi-kernel support vector machine for information fusion.” Adv. Eng. Inf. 56 (Apr): 101979. https://doi.org/10.1016/j.aei.2023.101979.
Inoue, Y., and H. Nagayoshi. 2020. “Crack detection as a weakly-supervised problem: Towards achieving less annotation-intensive crack detectors.” In Proc., 2020 25th Int. Conf. on Pattern Recognition (ICPR), 65–72. New York: IEEE.
Jahanshahi, M. R., S. F. Masri, C. W. Padgett, and G. S. Sukhatme. 2011. “An innovative methodology for detection and quantification of cracks through incorporation of depth perception.” Mach. Vision Appl. 24 (2): 227–241. https://doi.org/10.1007/s00138-011-0394-0.
Ji, D. J., J. Park, and D.-H. Cho. 2019. “ConvAE: A new channel autoencoder based on convolutional layers and residual connections.” IEEE Commun. Lett. 23 (10): 1769–1772. https://doi.org/10.1109/LCOMM.2019.2930287.
Lee, J., H. S. Kim, N. Kim, E. M. Ryu, and J. W. Kang. 2019. “Learning to detect cracks on damaged concrete surfaces using two-branched convolutional neural network.” Sensors 19 (21): 4796.
Li, G., S. He, Y. Ju, and K. Du. 2014. “Long-distance precision inspection method for bridge cracks with image processing.” Autom. Constr. 41 (Mar): 83–95. https://doi.org/10.1016/j.autcon.2013.10.021.
Li, G., Q. Liu, W. Ren, W. Qiao, B. Ma, and J. Wan. 2021. “Automatic recognition and analysis system of asphalt pavement cracks using interleaved low-rank group convolution hybrid deep network and SegNet fusing dense condition random field.” Measurement 170 (Jan): 108693.
Li, P., H. Xia, B. Zhou, F. Yan, and R. Guo. 2022. “A method to improve the accuracy of pavement crack identification by combining a semantic segmentation and edge detection model.” Appl. Sci. 12 (9): 4719.
Lin, K., L. Wang, and Z. Liu. 2020. “End-to-end human pose and mesh reconstruction with transformers.” In Proc., IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 1954–1963. New York: IEEE.
Liu, C., X. Sun, C. Chen, P. L. Rosin, Y. Yan, L. Jin, and X. Peng. 2019a. “Multi-scale residual hierarchical dense networks for single image super-resolution.” IEEE Access 7 (Mar): 60572–60583. https://doi.org/10.1109/ACCESS.2019.2915943.
Liu, Y., and M. Gao. 2022. “Detecting cracks in concrete structures with the baseline model of the visual characteristics of images.” Comput.-Aided Civ. Infrastruct. Eng. 37 (14): 1891–1913. https://doi.org/10.1111/mice.12874.
Liu, Y., J. Yao, X. Lu, R. Xie, and L. Li. 2019b. “DeepCrack: A deep hierarchical feature learning architecture for crack segmentation.” Neurocomputing 338 (Jun): 139–153. https://doi.org/10.1016/j.neucom.2019.01.036.
Liu, Z., Y. Cao, Y. Wang, and W. Wang. 2019c. “Computer vision-based concrete crack detection using U-net fully convolutional networks.” Autom. Constr. 104 (Mar): 129–139. https://doi.org/10.1016/j.autcon.2019.04.005.
Luo, W., J. Sun, Y. Liao, and Z. Zhang. 2022. “Research on the real-time detection method for image processing–based civil structure crack.” Trait. Signal 39 (6): 2223–2228. https://doi.org/10.18280/ts.390638.
Nigam, R., and S. K. Singh. 2020. “Crack detection in a beam using wavelet transform and photographic measurements.” Structures 25 (May): 436–447. https://doi.org/10.1016/j.istruc.2020.03.010.
Qiao, W., Q. Liu, X. Wu, B. Ma, and G. Li. 2021. “Automatic pixel-level pavement crack recognition using a deep feature aggregation segmentation network with a scSE attention mechanism module.” Sensors 21 (9): 2902. https://doi.org/10.3390/s21092902.
Shi, Y., L. Cui, Z. Qi, F. Meng, and Z. Chen. 2016. “Automatic road crack detection using random structured forests.” IEEE Trans. Intell. Transp. Syst. 17 (12): 3434–3445.
Trejo, P., S. García, and S. Vincent. 2023. “A classification tree for modeling ground fractures from subsidence.” Appl. Sci. 13 (5): 3123. https://doi.org/10.3390/app13053123.
Ulyanov, D., A. Vedaldi, and V. Lempitsky. 2016. “Instance normalization: The missing ingredient for fast stylization.” Preprint, submitted July 27, 2016. https://arxiv.org/abs/1607.08022.
Wang, C., X. Hou, and Y. Liu. 2020. “Three-dimensional crack recognition by unsupervised machine learning.” Rock Mech. Rock Eng. 54 (2): 893–903. https://doi.org/10.1007/s00603-020-02287-w.
Wang, C., Y. Wu, and Z. Liu. 2016. “Hierarchical boosting for transfer learning with multi-source.” In Proc., Int. Conf. on Artificial Intelligence and Robotics and the Int. Conf. on Automation, Control and Robotics Engineering—ICAIR-CACRE ‘16, 1–5. New York: Association for Computing Machinery. https://doi.org/10.1145/2952744.2952756.
Wang, J., X. He, S. Faming, G. Lu, H. Cong, and Q. Jiang. 2021a. “A real-time bridge crack detection method based on an improved inception-resnet-v2 structure.” IEEE Access 9 (Jun): 93209–93223. https://doi.org/10.1109/ACCESS.2021.3093210.
Wang, W., E. Xie, X. Li, D. P. Fan, and L. Shao. 2021b. “Pyramid vision transformer: A versatile backbone for dense prediction without convolutions.” In Proc., IEEE/CVF Int. Conf. on Computer Vision, 568–578. New York: IEEE.
Wang, X., R. Girshick, A. Gupta, and K. He. 2017. “Non-local neural networks.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 7794–7803. New York: IEEE.
Wei, Y., H. Xiao, H. Shi, Z. Jie, J. Feng, and T. S. Huang. 2018. “Revisiting dilated convolution: A simple approach for weakly- and semi-supervised semantic segmentation.” In Proc., 2018 IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 7268–7277. New York: IEEE.
Xie, S., and Z. Tu. 2017. “Holistically-nested edge detection.” Int. J. Comput. Vis. 125 (1–3): 3–18. https://doi.org/10.1007/s11263-017-1004-z.
Yang, F., L. Zhang, S. Yu, D. Prokhorov, X. Mei, and H. Ling. 2020. “Feature pyramid and hierarchical boosting network for pavement crack detection.” IEEE Trans. Intell. Transp. Syst. 21 (4): 1525–1535. https://doi.org/10.1109/TITS.2019.2910595.
Zhang, K., H. D. Cheng, and B. Zhang. 2018. “Unified approach to pavement crack and sealed crack detection using preclassification based on transfer learning.” J. Comput. Civ. Eng. 32 (2): 04018001. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000736.
Zhang, L., J. Shen, and B. Zhu. 2020. “A research on an improved Unet-based concrete crack detection algorithm.” Struct. Health Monit. 20 (4): 1864–1879. https://doi.org/10.1177/1475921720940068.
Zhang, Q. L., and Y. B. Yang. 2021. “ResT: An efficient transformer for visual recognition.” In Vol. 34 of Proc., 35th Conf. on Neural Information Processing Systems (NeurIPS), 15475–15485. Red Hook, NY: Curran Associates.
Zou, Q., Y. Cao, Q. Li, Q. Mao, and S. Wang. 2012. “CrackTree: Automatic crack detection from pavement images.” Pattern Recognit. Lett. 33 (3): 227–238. https://doi.org/10.1016/j.patrec.2011.11.004.
Information & Authors
Information
Published In
Journal of Performance of Constructed Facilities
Volume 38 • Issue 4 • August 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Oct 25, 2023
Accepted: Mar 14, 2024
Published online: Jun 5, 2024
Published in print: Aug 1, 2024
Discussion open until: Nov 5, 2024
ASCE Technical Topics:
- Bridge engineering
- Bridges
- Business management
- Computer vision and image processing
- Continuum mechanics
- Cracking
- Engineering fundamentals
- Engineering mechanics
- Fracture mechanics
- Information management
- Infrastructure
- Methodology (by type)
- Practice and Profession
- Public administration
- Public health and safety
- Safety
- Solid mechanics
- Structural engineering
- Structural systems
- Windows
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.