Multidamage Identification in High-Resolution Concrete Bridge Component Imagery Based on Deep Learning
Publication: Journal of Performance of Constructed Facilities
Volume 38, Issue 5
Abstract
The identification of damage to concrete bridge surfaces is of great significance to maintaining the durability and reliability of bridges. However, it is difficult to identify small areas of damage, especially in high-resolution images. To conduct damage identification research in a targeted manner, this paper proposes a method of multidamage identification in high-resolution images based on You Only Look Once version 5 (YOLOv5). In this paper, a data set labeled with four types of damage (crack, spallation, hole, and rebar) is used for training, validation, and testing. To ensure that the network adequately learns the damage features, the high-resolution images are cropped into subimages via the autoadaptive window cropping method (AWCM) proposed in this paper. The cropping method can crop images according to the label information and protect the damage features from destruction during the cropping process. To avoid overfitting, it is necessary to balance the volume of different types of damage. After balancing, the count for each category is as follows: crack (4,980), spallation (5,225), hole (5,211) and rebar (5,020). The balanced data set can be used to train the deep learning network and construct the multidamage identification model. After identification, the subimages, and the prediction boxes of damages in them are restored to their original high-resolution images. The results show that the mean average precision (mAP) of all classes is 94.2%, and the values for cracks, spallation, holes, and rebars are 86.9%, 98.1%, 92.3%, and 99.4%, respectively, which indicates that the proposed method outperforms the other three methods (inputting original images directly, the sliding window cropping method, and the random centroid cropping method).
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All the data, models, and code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This research work was jointly supported by the National Natural Science Foundation of China (Grants Nos. 52250011 and 12002224), and the Fundamental Research Funds for the Central Universities (Grant Nos. DUT22ZD213 and DUT22QN235).
References
Abdel-Qader, I., O. Abudayyeh, and M. E. Kelly. 2003. “Analysis of edge-detection techniques for crack identification in bridges.” J. Comput. Civ. Eng. 17 (4): 255–263. https://doi.org/10.1061/(ASCE)0887-3801(2003)17:4(255).
Arulalan, V., and D. Kumar. 2023. “Efficient object detection and classification approach using HTYOLOV4 and M2RFO-CNN.” Comput. Syst. Sci. Eng. 44 (2): 1703–1717. https://doi.org/10.32604/csse.2023.026744.
Baduge, S. K., S. Thilakarathna, J. S. Perera, M. Arashpour, P. Sharafi, B. Teodosio, A. Shringi, and P. Mendis. 2022. “Artificial intelligence and smart vision for building and construction 4.0: Machine and deep learning methods and applications.” Autom. Constr. 141 (Sep): 104440. https://doi.org/10.1016/j.autcon.2022.104440.
Cao, M. T., N. M. Nguyen, K. T. Chang, X. L. Tran, and N. D. Hoang. 2021. “Automatic recognition of concrete spall using image processing and metaheuristic optimized LogitBoost classification tree.” Adv. Eng. Software 159 (Sep): 103031. https://doi.org/10.1016/j.advengsoft.2021.103031.
Cha, Y. J., W. Choi, G. Suh, S. Mahmoudkhani, and O. Buyukozturk. 2018. “Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types.” Comput.-Aided Civ. Infrastruct. Eng. 33 (9): 731–747. https://doi.org/10.1111/mice.12334.
Chen, L., W. Chen, L. Wang, C. Zhai, X. Hu, L. Sun, Y. Tian, X. Huang, and L. Jiang. 2023. “Convolutional neural networks (CNNs)-based multi-category damage detection and recognition of high-speed rail (HSR) reinforced concrete (RC) bridges using test images.” Eng. Struct. 276 (Feb): 115306. https://doi.org/10.1016/j.engstruct.2022.115306.
Deng, W., Y. Mou, T. Kashiwa, S. Escalera, K. Nagai, K. Nakayama, Y. Matsuo, and H. Prendinger. 2020. “Vision based pixel-level bridge structural damage detection using a link ASPP network.” Autom. Constr. 110 (Feb): 102973. https://doi.org/10.1016/j.autcon.2019.102973.
Du, Y. L., T. H. Yi, X. J. Li, X. L. Rong, L. J. Dong, D. W. Wang, G. Yang, and Z. Leng. 2023. “Advances in intellectualization of transportation infrastructures.” Engineering 23 (4): 64. https://doi.org/10.1016/j.eng.2023.01.011.
Fan, C. L., and Y. J. Chung. 2022. “Supervised machine learning-based detection of concrete efflorescence.” Symmetry 14 (11): 2384. https://doi.org/10.3390/sym14112384.
Hoang, N. D., T. C. Huynh, and V. D. Tran. 2021. “Concrete spalling severity classification using image texture analysis and a novel jellyfish search pptimized machine learning approach.” Adv. Civ. Eng. 2021: 5551555. https://doi.org/10.1155/2021/5551555.
Inam, H., N. Ul Islam, M. U. Akram, and F. Ullah. 2023. “Smart and automated infrastructure management: A deep learning approach for crack detection in bridge images.” Sustainability 15 (3): 1866. https://doi.org/10.3390/su15031866.
Leach, S., Y. Xue, R. Sridhar, S. Paal, Z. Wang, and R. Murphy. 2021. “Data augmentation for improving deep learning models in building inspections or postdisaster evaluation.” J. Perform. Constr. Facil. 35 (4): 04021029. https://doi.org/10.1061/(ASCE)CF.1943-5509.0001594.
Maeda, H., Y. Sekimoto, T. Seto, T. Kashiyama, and H. Omata. 2018. “Road damage detection and classification using deep neural networks with smartphone images.” Comput.-Aided Civ. Infrastruct. Eng. 33 (12): 1127–1141. https://doi.org/10.1111/mice.12387.
Mir, B. A., T. Sasaki, K. Nakao, K. Nagae, K. Nakada, M. Mitani, T. Tsukada, N. Osada, K. Terabayashi, and M. Jindai. 2022. “Machine learning-based evaluation of the damage caused by cracks on concrete structures.” Precis. Eng. 76 (Jul): 314–327. https://doi.org/10.1016/j.precisioneng.2022.03.016.
Mukhiddinov, M., A. B. Abdusalomov, and J. Cho. 2022. “A wildfire smoke detection system using unmanned aerial vehicle images based on the optimized YOLOv5.” Sensors 22 (23): 9384. https://doi.org/10.3390/s22239384.
Nishikawa, T., J. Yoshida, T. Sugiyama, and Y. Fujino. 2012. “Concrete crack detection by multiple sequential image filtering.” Comput.-Aided Civ. Infrastruct. Eng. 27 (1): 29–47. https://doi.org/10.1111/j.1467-8667.2011.00716.x.
Qiu, Z., Z. Zhao, S. Chen, J. Zeng, Y. Huang, and B. Xiang. 2022. “Application of an improved YOLOv5 algorithm in real-time detection of foreign objects by ground penetrating radar.” Remote Sens. 14 (8): 1895. https://doi.org/10.3390/rs14081895.
Su, S. L., and T. Gong. 2020. “Bridge pavement crack detection under uneven illumination using improved PCNN algorithm.” Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XLII-3/W10: 1033–1040. https://doi.org/10.1155/2021/5538573.
Sun, L., Z. Shang, Y. Xia, S. Bhowmick, and S. Nagarajaiah. 2020. “Review of bridge structural health monitoring aided by big data and artificial intelligence: From condition assessment to damage detection.” J. Struct. Eng. 146 (5): 04020073. https://doi.org/10.1061/(ASCE)ST.1943-541X.0002535.
Teng, S., Z. Liu, and X. Li. 2022. “Improved YOLOv3-based bridge surface defect detection by combining high- and low-resolution feature images.” Buildings 12 (8): 1225. https://doi.org/10.3390/buildings12081225.
Wan, H., L. Gao, Z. Yuan, H. Qu, Q. Sun, H. Cheng, and R. Wang. 2023. “A novel transformer model for surface damage detection and cognition of concrete bridges.” Expert Syst. Appl. 213 (Part B): 119019. https://doi.org/10.1016/j.eswa.2022.119019.
Wang, C., X. Hou, and Y. Liu. 2021. “Three-dimensional crack recognition by unsupervised machine learning.” Rock Mech. Rock Eng. 54 (2): 893–903. https://doi.org/10.1007/s00603-020-02287-w.
Wang, D., M. Zhang, D. Sheng, and W. Chen. 2023. “Bolt positioning detection based on improved YOLOv5 for bridge structural health monitoring.” Sensors 23 (1): 396. https://doi.org/10.3390/s23010396.
Wang, H. F., L. Zhai, H. Huang, L. M. Guan, K. N. Mu, and G. Wang. 2020. “Measurement for cracks at the bottom of bridges based on tethered creeping unmanned aerial vehicle.” Autom. Constr. 119 (Nov): 103330. https://doi.org/10.1016/j.autcon.2020.103330.
Wang, W., A. Zhang, K. C. P. Wang, A. F. Braham, and S. Qiu. 2018. “Pavement crack width measurement based on Laplace’s equation for continuity and unambiguity.” Comput.-Aided Civ. Infrastruct. Eng. 33 (2): 110–123. https://doi.org/10.1111/mice.12319.
Wang, Y., J. Y. Zhang, J. X. Liu, Y. Zhang, Z. P. Chen, C. G. Li, K. He, and R. B. Yan. 2019. “Research on crack detection algorithm of the concrete bridge based on image processing.” Procedia Comput. Sci. 154 (Jan): 610–616. https://doi.org/10.1016/j.procs.2019.06.096.
Wu, F., J. Duan, P. Ai, Z. Chen, Z. Yang, and X. Zou. 2022. “Rachis detection and three-dimensional localization of cut off point for vision-based banana robot.” Comput. Electron. Agric. 198 (Jul): 107079. https://doi.org/10.1016/j.compag.2022.107079.
Xie, R., J. Yao, K. Liu, X. Lu, Y. Liu, M. Xia, and Q. Zeng. 2018. “Automatic multi-image stitching for concrete bridge inspection by combining point and line features.” Autom. Constr. 90 (Jun): 265–280. https://doi.org/10.1016/j.autcon.2018.02.021.
Xu, Y., and J. M. W. Brownjohn. 2018. “Review of machine-vision based methodologies for displacement measurement in civil structures.” J. Civ. Struct. Health Monit. 8 (1): 91–110. https://doi.org/10.1007/s13349-017-0261-4.
Xu, Y., S. Wei, Y. Bao, and H. Li. 2019. “Automatic seismic damage identification of reinforced concrete columns from images by a region-based deep convolutional neural network.” Struct. Control. Health Monit. 26 (3): e2313. https://doi.org/10.1002/stc.2313.
Yu, J. C., T. H. Yi, S. H. Zhang, H. N. Li, Y. F. Wang, and X. D. Mei. 2023. “Automatic quantitative identification of bridge surface cracks based on deep learning.” J. Perform. Constr. Facil. 37 (1): 04022072. https://doi.org/10.1061/JPCFEV.CFENG-4238.
Yu, L., S. He, X. Liu, S. Jiang, and S. Xiang. 2022. “Intelligent crack detection and quantification in the concrete bridge: A deep learning-assisted image processing approach.” Adv. Civ. Eng. 2022: 1813821. https://doi.org/10.1155/2022/1813821.
Zhang, C., C. Chang, and M. Jamshidi. 2020. “Concrete bridge surface damage detection using a single-stage detector.” Comput.-Aided Civ. Infrastruct. Eng. 35 (4): 389–409. https://doi.org/10.1111/mice.12500.
Zhang, Y. C., T. H. Yi, S. Lin, H. N. Li, and S. Lv. 2022. “Automatic corrosive environment detection of RC bridge decks from ground-penetrating radar data based on deep learning.” J. Perform. Constr. Facil. 36 (2): 04022011. https://doi.org/10.1061/(ASCE)CF.1943-5509.0001712.
Zhao, S., F. Kang, and J. Li. 2022. “Concrete dam damage detection and localisation based on YOLOv5s-HSC and photogrammetric 3D reconstruction.” Autom. Constr. 143 (Nov): 104555. https://doi.org/10.1016/j.autcon.2022.104555.
Zhao, Z., T. Liu, and X. Zhao. 2021. “Variable selection from image texture feature for automatic classification of concrete surface voids.” Comput. Intell. Neurosci. 2021: 5538573. https://doi.org/10.1155/2021/5538573.
Information & Authors
Information
Published In
Journal of Performance of Constructed Facilities
Volume 38 • Issue 5 • October 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Sep 12, 2023
Accepted: Mar 27, 2024
Published online: Jun 28, 2024
Published in print: Oct 1, 2024
Discussion open until: Nov 28, 2024
ASCE Technical Topics:
- Architectural engineering
- Artificial intelligence and machine learning
- Bridge components
- Bridge engineering
- Bridge management
- Bridges
- Bridges (by material)
- Building management
- Computer programming
- Computing in civil engineering
- Concrete bridges
- Continuum mechanics
- Corrosion
- Cracking
- Deterioration
- Engineering mechanics
- Fracture mechanics
- Maintenance and operation
- Materials characterization
- Materials engineering
- Neural networks
- Solid mechanics
- Spalling
- Structural engineering
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.