Automated Geometric Quantification of Building Exterior Wall Cracks Based on Computer Vision
Publication: Journal of Performance of Constructed Facilities
Volume 38, Issue 4
Abstract
Crack detection methods of high-rise building walls based on traditional computer vision (CV) heavily rely on manual selection and extraction of design features. Convolutional neural network (CNN)-based CV can actively learn the features of cracks and adapt to complex backgrounds, solving the limitations of traditional crack detection methods. This paper explores faster region-CNN, single shot multibox detector (SSD), You Only Look Once for crack detection, and Mask R-CNN for crack segmentation and proposes a novel automatic crack geometric quantification method by combining CNN-based object detection and segmentation. The contents include (1) crack detection and bounding box extraction, exploring a variety of models, selecting the best model to detect the image taken by an unmanned aerial vehicle (UAV), and extracting the crack region; (2) crack segmentation, using the detection results of the first part as input for more accurate detection and segmentation of cracks; and (3) a novel pixel-level geometric quantization method of crack based on Hough straight-line detection, mainly including crack length and width. Then, the pixel level is transformed into the actual geometric quantization to simply determine the crack severity. The three models generated in these three parts can be used for managing exterior wall cracks in high-rise buildings for different inspection purposes.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This work was financially supported by the Foundation for Distinguished Young Talents in Higher Education of Guangdong, China (FDYT), No. 2020KQNCX060. The authors wish to express their gratitude for this financial support.
References
Bang, H., J. Min, and H. Jeon. 2021. “Deep learning-based concrete surface damage monitoring method using structured lights and depth camera.” Sensors 21 (8): 2759. https://doi.org/10.3390/s21082759.
Beskopylny, A. N., E. M. Shcherban, S. A. Stel’makh, L. R. Mailyan, B. Meskhi, I. Razveeva, A. Kozhakin, N. Beskopylny, D. El’shaeva, and S. Artamonov. 2023. “Method for concrete structure analysis by microscopy of hardened cement paste and crack segmentation using a convolutional neural network.” J. Compos. Sci. 7 (May): 19. https://doi.org/10.3390/jcs7080327.
Bhowmick, S., S. Nagarajaiah, and A. Veeraraghavan. 2020. “Vision and deep learning-based algorithms to detect and quantify cracks on concrete surfaces from UAV videos.” Sensors 20 (Mar): 6299. https://doi.org/10.3390/s20216299.
CECS (China Association for Engineering Construction Standardization). 2011. Technical specification for inspection and treatment of cracks in buildings. CECS 293. Beijing: China Association for Engineering Construction Standardization.
Cevallos-Torres, L. J., D. M. Gilces, A. Guijarro-Rodriguez, R. Barriga-Diaz, M. Leyva-Vazquez, and M. Botto-Tobar. 2019. “An approach to the detection of post-seismic structural damage based on image segmentation methods.” In Technology trends, edited by M. BottoTobar, G. Pizarro, M. ZunigaPrieto, M. Darmas, and M. Z. Sanchez, Berlin: Springer-Verlag.
Chen, X. D., C. Fu, M. Tie, C. W. Sham, and H. F. Ma. 2023. “AFFNet: An attention-based feature-fused network for surface defect segmentation.” Appl. Sci. 13 (Mar): 16. https://doi.org/10.3390/app13116428.
Cheng, J. C., and M. Wang. 2018. “Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques.” Autom. Constr. 95 (May): 155–171. https://doi.org/10.1016/j.autcon.2018.08.006.
CTBUH (Council on Tall Buildings and Urban Habitat). 2018. “Council on tall buildings and urban habitat.” Accessed April 7, 2021. https://www.ctbuh.org/.
Fawzy, H. E. 2019. “3D laser scanning and close-range photogrammetry for buildings documentation: A hybrid technique towards a better accuracy.” Alexandria Eng. J. 58 (Mar): 1191–1204. https://doi.org/10.1016/j.aej.2019.10.003.
Flah, M., A. R. Suleiman, and M. L. Nehdi. 2020. “Classification and quantification of cracks in concrete structures using deep learning image-based techniques.” Cem. Concr. Compos. 114 (May): 19. https://doi.org/10.1016/j.cemconcomp.2020.103781.
Gaur, A., K. Kishore, R. Jain, A. Pandey, P. Singh, N. K. Wagri, and A. B. Roy-Chowdhury. 2023. “A novel approach for industrial concrete defect identification based on image processing and deep convolutional neural networks.” Case Stud. Constr. Mater. 19 (May): e02392. https://doi.org/10.1016/j.cscm.2023.e02392.
Gopalakrishnan, K., S. K. Khaitan, A. Choudhary, and A. Agrawal. 2017. “Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection.” Constr. Build. Mater. 157 (Dec): 322–330. https://doi.org/10.1016/j.conbuildmat.2017.09.110.
Guo, Y. H., X. S. Shen, J. Linke, Z. H. Wang, and K. Barati. 2023. “Quantification of structural defects using pixel level spatial information from photogrammetry.” Sensors 23 (13): 5878. https://doi.org/10.3390/s23135878.
He, K., G. Gkioxari, P. Dollár, and R. Girshick. 2020. “Mask R-CNN.” IEEE Trans. Pattern Anal. Mach. Intell. 42 (Apr): 386–397. https://doi.org/10.1109/TPAMI.2018.2844175.
He, Y., Z. H. Jin, J. Q. Zhang, S. Teng, G. F. Chen, X. L. Sun, and F. S. Cui. 2022. “Pavement surface defect detection using mask region-based convolutional neural networks and transfer learning.” Appl. Sci. 12 (Feb): 20. https://doi.org/10.3390/app12157364.
Hu, G. X., B. L. Hu, Z. Yang, L. Huang, and P. Li. 2021. “Pavement crack detection method based on deep learning models.” Wireless Commun. Mobile Comput. 2021 (May): 13. https://doi.org/10.1155/2021/5573590.
Hu, X. J., J. Yang, F. L. Jiang, A. Hussain, K. Dashtipour, and M. Gogate. 2023. “Steel surface defect detection based on self-supervised contrastive representation learning with matching metric.” Appl. Soft Comput. 145 (Jun): 21. https://doi.org/10.1016/j.asoc.2023.110578.
Ji, A., X. Xue, Y. Wang, X. Luo, and W. Xue. 2020. “An integrated approach to automatic pixel-level crack detection and quantification of asphalt pavement.” Autom. Constr. 114 (Jun): 103176. https://doi.org/10.1016/j.autcon.2020.103176.
Jocher, G., et al. 2020. Ultralytics/yolov5: v5.0—YOLOv5-P6 1280 models, AWS, Supervise.ly and YouTube integrations (v5.0). Geneva: Zenodo. https://doi.org/10.5281/zenodo.4679653.
Kim, B., and S. Cho. 2019. “Image-based concrete crack assessment using mask and region-based convolutional neural network.” Struct. Control Health Monit. 26 (8): 15. https://doi.org/10.1002/stc.2381.
Kingma, D. P., and J. Ba. 2014. “Adam: A method for stochastic optimization.” Preprint, submitted December 14, 2014. http://arxiv.org/abs/1412.6980.
Kumar, P., S. Batchu, S. N. Swamy, and S. R. Kota. 2021. “Real-time concrete damage detection using deep learning for high rise structures.” IEEE Access 9 (May): 112312–112331. https://doi.org/10.1109/ACCESS.2021.3102647.
Kumar, S. S., M. Z. Wang, D. M. Abraham, M. R. Jahanshahi, T. Iseley, and J. C. P. Cheng. 2020. “Deep learning-based automated detection of sewer defects in CCTV videos.” J. Comput. Civ. Eng. 34 (1): 13. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000866.
Li, C. S., D. Zhang, S. S. Du, and B. Shi. 2016. “Computed tomography based numerical simulation for triaxial test of soil-rock mixture.” Comput. Geotech. 73 (Mar): 179–188. https://doi.org/10.1016/j.compgeo.2015.12.005.
Li, D. W., Q. Xie, X. X. Gong, Z. H. Yu, J. X. Xu, Y. X. Sun, and J. Wang. 2021a. “Automatic defect detection of metro tunnel surfaces using a vision-based inspection system.” Adv. Eng. Inf. 47 (Mar): 101206. https://doi.org/10.1016/j.aei.2020.101206.
Li, J., C. Yuan, and X. Wang. 2023a. “Real-time instance-level detection of asphalt pavement distress combining space-to-depth (SPD) YOLO and omni-scale network (OSNet).” Autom. Constr. 155 (Nov): 105062. https://doi.org/10.1016/j.autcon.2023.105062.
Li, R., J. Yu, F. Li, R. Yang, Y. Wang, and Z. Peng. 2023b. “Automatic bridge crack detection using unmanned aerial vehicle and faster R-CNN.” Constr. Build. Mater. 362 (Jan): 129659. https://doi.org/10.1016/j.conbuildmat.2022.129659.
Li, S. W., X. Y. Gu, X. R. Xu, D. W. Xu, T. J. Zhang, Z. Liu, and Q. Dong. 2021b. “Detection of concealed cracks from ground penetrating radar images based on deep learning algorithm.” Constr. Build. Mater. 273 (Dec): 14. https://doi.org/10.1016/j.conbuildmat.2020.121949.
Lin, Y. C., W. H. Chen, and C. H. Kuo. 2021. “Implementation of pavement defect detection system on edge computing platform.” Appl. Sci. 11 (8): 16. https://doi.org/10.3390/app11083725.
Liu, C., C. S. Tang, B. Shi, and W. B. Suo. 2013. “Automatic quantification of crack patterns by image processing.” Comput. Geosci. 57 (Aug): 77–80. https://doi.org/10.1016/j.cageo.2013.04.008.
Liu, W., D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, and A. C. Berg. 2015. “SSD: Single shot multibox detector.” Preprint, submitted April 23, 2018. http://arxiv.org/abs/1512.02325.
Manjunatha, P., S. F. Masri, A. Nakano, and L. C. Wellford. 2023. “CrackDenseLinkNet: A deep convolutional neural network for semantic segmentation of cracks on concrete surface images.” Struct. Health Monit. 23 (2): 796–817. https://doi.org/10.1177/14759217231173305.
Otsu, N. 1979. “Threshold selection method from gray-level histograms.” IEEE Transactions Syst. Man Cybern. 9 (May): 62–66. https://doi.org/10.1109/TSMC.1979.4310076.
Qiu, S., W. Wang, S. Wang, and K. C. Wang. 2017. “Methodology for accurate AASHTO PP67-10–Based cracking quantification using 1-mm 3D pavement images.” J. Comput. Civ. Eng. 31 (2): 04016056. https://doi.org/10.1061/(asce)cp.1943-5487.0000627.
Ramalingam, B., R. E. Mohan, S. Pookkuttath, B. F. Gomez, T. W. Sairam Borusu, T. Wee Teng, and Y. K. Tamilselvam. 2020. “Remote insects trap monitoring system using deep learning framework and IoT.” Sensors 20 (18): 16. https://doi.org/10.3390/s20185280.
Redmon, J., and A. Farhadi. 2018. “YOLOv3: An incremental improvement.” Preprint, submitted April 8, 2018. http://arxiv.org/abs/1804.02767.
Ren, M., X. Zhang, X. Chen, B. Zhou, and Z. Feng. 2023. “YOLOv5s-M: A deep learning network model for road pavement damage detection from urban street-view imagery.” Int. J. Appl. Earth Obs. Geoinf. 120 (Jun): 103335. https://doi.org/10.1016/j.jag.2023.103335.
Ren, S. Q., K. M. He, R. Girshick, and J. Sun. 2017. “Faster R-CNN: Towards real-time object detection with region proposal networks.” IEEE Trans. Pattern Anal. Mach. Intell. 39 (Mar): 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031.
Silva, L. A., H. S. San Blas, D. P. Garcia, A. S. Mendes, and G. V. Gonzalez. 2020. “An architectural multi-agent system for a pavement monitoring system with pothole recognition in UAV images.” Sensors 20 (May): 23. https://doi.org/10.3390/s20216205.
Song, H. 2017. “Design of building surface crack detection system based on embedded.” Master thesis, Dept. of Instrumentation Engineering, China Jiliang Univ.
Tan, Y., R. Y. Cai, J. R. Li, P. L. Chen, and M. Z. Wang. 2021a. “Automatic detection of sewer defects based on improved you only look once algorithm.” Autom. Constr. 131 (Jun): 17. https://doi.org/10.1016/j.autcon.2021.103912.
Tan, Y., S. Li, H. Liu, P. Chen, and Z. Zhou. 2021b. “Automatic inspection data collection of building surface based on BIM and UAV.” Autom. Constr. 131 (Nov): 103881. https://doi.org/10.1016/j.autcon.2021.103881.
Tang, J., Z. Wang, and X. Zhang. 2011. Linear detection technology based on Hough transform, 33–35. Beijing: Scientific and Technological Information.
Teng, S., and G. F. Chen. 2022. “Deep convolution neural network-based crack feature extraction, detection and quantification.” J. Fail. Anal. Prev. 22 (Feb): 1308–1321. https://doi.org/10.1007/s11668-022-01430-9.
Theiner, Y., and G. Hofstetter. 2009. “Numerical prediction of crack propagation and crack widths in concrete structures.” Eng. Struct. 31 (8): 1832–1840. https://doi.org/10.1016/j.engstruct.2009.02.041.
Tran, T. S., V. P. Tran, H. J. Lee, J. M. Flores, and V. P. Le. 2022. “A two-step sequential automated crack detection and severity classification process for asphalt pavements.” Int. J. Pavement Eng. 23 (6): 2019–2033. https://doi.org/10.1080/10298436.2020.1836561.
Wang, N. N., Q. G. Zhao, S. Y. Li, X. F. Zhao, and P. Zhao. 2018. “Damage classification for masonry historic structures using convolutional neural networks based on still images.” Comput.-Aided Civ. Infrastruct. Eng. 33 (12): 1073–1089. https://doi.org/10.1111/mice.12411.
Wang, R., T. Qi, S. Hu, and Y. Wan. 2017. “Background processing and breakpoint connection algorithm in crack detection of tunnel lining.” J. Appl. Basic Eng. Sci. 25 (4): 742–750. https://doi.org/10.16058/j.issn.1005-0930.2017.04.009.
Wang, Z., G. S. Xu, Y. Ding, B. Wu, and G. Y. Lu. 2020. “A vision-based active learning convolutional neural network model for concrete surface crack detection.” Adv. Struct. Eng. 23 (Nov): 2952–2964. https://doi.org/10.1177/1369433220924792.
Yang, C., J. J. Chen, Z. Y. Li, and Y. Huang. 2021. “Structural crack detection and recognition based on deep learning.” Appl. Sci. 11 (6): 13. https://doi.org/10.3390/app11010013.
Yao, G., Y. J. Sun, M. P. Wong, and X. N. Lv. 2021. “A real-time detection method for concrete surface cracks based on improved YOLOv4.” Symmetry 13 (9): 16. https://doi.org/10.3390/sym13091716.
Ye, H., G. Yu, B. Zhang, and T. Guo. 2012. “Development of portable digital detector for building surface crack.” Mach. Tool Hydraul. 40 (Dec): 6–8. https://doi.org/10.3969/j.issn.1001-3881.2012.18.002.
Yin, X. F., Y. Chen, A. Bouferguene, H. Zaman, M. Al-Hussein, and L. Kurach. 2020. “A deep learning-based framework for an automated defect detection system for sewer pipes.” Autom. Constr. 109 (Jan): 102967. https://doi.org/10.1016/j.autcon.2019.102967.
Yu, A. B., W. S. Mei, and M. L. Han. 2021. “Deep learning based method of longitudinal dislocation detection for metro shield tunnel segment.” Tunnelling Underground Space Technol. 113 (Apr): 103949. https://doi.org/10.1016/j.tust.2021.103949.
Zhang, C., J. Cui, J. G. Wu, and X. Zhang. 2023. “Attention mechanism and texture contextual information for steel plate defects detection.” J. Intell. Manuf. 22 (3): 1–22. https://doi.org/10.1007/s10845-023-02149-6.
Zhang, C. B., C. C. Chang, and M. Jamshidi. 2020. “Concrete bridge surface damage detection using a single-stage detector.” Comput.-Aided Civ. Infrastruct. Eng. 35 (May): 389–409. https://doi.org/10.1111/mice.12500.
Zhao, S., F. Kang, and J. Li. 2022. “Concrete dam damage detection and localisation based on YOLOv5s-HSC and photogrammetric 3D reconstruction.” Autom. Constr. 143 (Nov): 104555. https://doi.org/10.1016/j.autcon.2022.104555.
Zhou, S. L., and W. Song. 2020. “Deep learning-based roadway crack classification using laser-scanned range images: A comparative study on hyperparameter selection.” Autom. Constr. 114 (Jun): 103171. https://doi.org/10.1016/j.autcon.2020.103171.
Zhu, J. S., and J. B. Song. 2020. “An intelligent classification model for surface defects on cement concrete bridges.” Appl. Sci. 10 (3): 15. https://doi.org/10.3390/app10030972.
Information & Authors
Information
Published In
Journal of Performance of Constructed Facilities
Volume 38 • Issue 4 • August 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Jul 15, 2023
Accepted: Jan 23, 2024
Published online: Apr 27, 2024
Published in print: Aug 1, 2024
Discussion open until: Sep 27, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.