Semantic Segmentation of Defects in Infrared Thermographic Images of Highly Damaged Concrete Structures
Publication: Journal of Performance of Constructed Facilities
Volume 35, Issue 1
Abstract
There is a global research trend to enhance condition assessment of the concrete infrastructure by the development of advanced nondestructive testing (NDT) methods. Computer vision–based systems have been developed to detect different types of defects in both regular and thermographic images because these systems could offer a timely and cost-effective solution and are able to tackle the inconsistency issues of manual assessment. This paper investigates the performance of different deep neural network models to detect main concrete anomalies, including delamination, cracks, spalling, and patches in thermographic and regular images captured from a variety of distances and viewpoints. These models were trained and tested using images taken from a century-old buttress dam and validated in images captured from the decks of two concrete bridges. The results showed that the MobileNetV2 had promising performance in the identification of multiclass damages in the thermal images, identifying 79.7% of the total delamination, cracks, spalling, and patches on the test images of highly damaged concrete areas. The VGG 16 model showed better precision by reducing the number of false detections.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or codes that support the findings of this study are available from the corresponding author upon reasonable request. (Digital and thermal images and MATLAB codes.)
Acknowledgments
The authors would like to thank Global Affairs Canada and Lakehead University for financial support.
References
An, Y. K., K. Jang, B. Kim, and S. Cho. 2018. “Deep learning-based concrete crack detection using hybrid images.” In Sensors and smart structures technologies for civil, mechanical, and aerospace systems 2018, 1059812. Bellingham, WA: International Society for Optics and Photonics. https://doi.org/10.1117/12.2294959.
Arcadis Canada. 2018. “Environmental study report for the rehabilitation of the Boulevard Lake Dam.” Accessed February 29, 2020. https://www.thunderbay.ca/en/city-hall/resources/351254---ESR-For-Boulevard-Lake-Dam-Rehabilitation-11Dec2018-Final.pdf.
ASCE Infrastructure Report Card. 2017. “America’s infrastructure report card 2017.” Accessed February 11, 2020. https://www.infrastructurereportcard.org/.
ASTM. 2013. Standard test method for detecting delaminations in bridge decks using infrared thermography. ASTM D4788-03. West Conshohocken, PA: ASTM.
Beckman, G. H., D. Polyzois, and Y. J. Cha. 2019. “Deep learning-based automatic volumetric damage quantification using depth camera.” Autom. Constr. 99 (Mar): 114–124. https://doi.org/10.1016/j.autcon.2018.12.006.
Canadian Infrastructure Report Card. 2019. “Assessing the health of our communities’ infrastructure.” Accessed February 11, 2020. http://www.canadainfrastructure.ca.
Cheng, C., R. Na, and Z. Shen. 2019a. “Thermographic Laplacian-pyramid filtering to enhance delamination detection in concrete structure.” Infrared Phys. Technol. 97 (Mar): 162–176. https://doi.org/10.1016/j.infrared.2018.12.039.
Cheng, C., Z. Shang, and Z. Shen. 2019b. “CNN-based deep architecture for reinforced concrete delamination segmentation through thermography.” In Proc., ASCE Int. Conf. on Computing in Civil Engineering. Reston, VA: ASCE. https://doi.org/10.1061/9780784482445.007.
Cheng, C., and Z. Shen. 2020. “The application of gray-scale level-set method in segmentation of concrete deck delamination using infrared images.” Constr. Build. Mater. 240 (Apr): 117974. https://doi.org/10.1016/j.conbuildmat.2019.117974.
Chollet, F. 2017. “Xception: Deep learning with depthwise separable convolutions.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 1251–1258. New York: IEEE. https://doi.org/10.1109/CVPR.2017.195.
Dawood, T., Z. Zhu, and T. Zayed. 2017. “Machine vision-based model for spalling detection and quantification in subway networks.” Autom. Constr. 81 (Sep): 149–160. https://doi.org/10.1016/j.autcon.2017.06.008.
Deng, J., Y. Lu, and V. C. S. Lee. 2020. “Concrete crack detection with handwriting script interferences using faster region-based convolutional neural network.” Comput.-Aided Civ. Infrastruct. Eng. 35 (4): 373–388. https://doi.org/10.1111/mice.12497.
Dung, C. V., and L. D. Anh. 2019. “Autonomous concrete crack detection using deep fully convolutional neural network.” Autom. Constr. 99 (Mar): 52–58. https://doi.org/10.1016/j.autcon.2018.11.028.
Everingham, M., L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. 2012. “The PASCAL visual object classes challenge 2012 (VOC2012) results.” Accessed March 10, 2020. http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html.
Goodfellow, I., Y. Bengio, and A. Courville. 2016. “Deep learning.” Accessed March 8, 2020. https://www.deeplearningbook.org/.
He, K., X. Zhang, S. Ren, and J. Sun. 2016. “Deep residual learning for image recognition.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 770–778. New York: IEEE. https://doi.org/10.1109/CVPR.2016.90.
Hiasa, S., R. Birgul, and F. N. Catbas. 2016. “Infrared thermography for civil structural assessment: Demonstrations with laboratory and field studies.” J. Civ. Struct. Health Monit. 6 (3): 619–636. https://doi.org/10.1007/s13349-016-0180-9.
Kabir, S. 2010. “Imaging-based detection of AAR induced map-crack damage in concrete structure.” NDT and E Int. 43 (6): 461–469. https://doi.org/10.1016/j.ndteint.2010.04.007.
Kong, X., and J. Li. 2019. “Non-contact fatigue crack detection in civil infrastructure through image overlapping and crack breathing sensing.” Autom. Constr. 99 (Mar): 125–139. https://doi.org/10.1016/j.autcon.2018.12.011.
Kumar, S. S., D. M. Abraham, M. R. Jahanshahi, T. Iseley, and J. Starr. 2018. “Automated defect classification in sewer closed circuit television inspections using deep convolutional neural networks.” Autom. Constr. 91 (Jul): 273–283. https://doi.org/10.1016/j.autcon.2018.03.028.
Kumar, S. S., M. Wang, D. M. Abraham, M. R. Jahanshahi, T. Iseley, and J. C. Cheng. 2020. “Deep learning–based automated detection of sewer defects in CCTV videos.” J. Comput. Civ. Eng. 34 (1): 04019047. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000866.
Liu, L., W. Ouyang, X. Wang, P. Fieguth, J. Chen, X. Liu, and M. Pietikäinen. 2020. “Deep learning for generic object detection: A survey.” Int. J. Comput. Vision 128 (2): 261–318. https://doi.org/10.1007/s11263-019-01247-4.
Liu, Z., Y. Cao, Y. Wang, and W. Wang. 2019. “Computer vision-based concrete crack detection using U-net fully convolutional networks.” Autom. Constr. 104 (Aug): 129–139. https://doi.org/10.1016/j.autcon.2019.04.005.
McLaughlin, E., N. Charron, and S. Narasimhan. 2019. “Combining deep learning and robotics for automated concrete delamination assessment.” In Vol. 36 of Proc., Int. Symp. on Automation and Robotics in Construction, 485–492. Oulu, Finland: International Association for Automation and Robotics. https://doi.org/10.22260/ISARC2019/0065.
McLaughlin, E., N. Charron, and S. Narasimhan. 2020. “Automated defect quantification in concrete bridges using robotics and deep learning.” J. Comput. Civ. Eng. 34 (5): 04020029. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000915.
Moghaddam, A. S., E. R. Azar, Y. Mejias, and H. Bell. 2020. “Estimating stripping of asphalt coating using k-means clustering and machine learning–based classification.” J. Comput. Civ. Eng. 34 (1): 04019044. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000864.
Omar, T., and M. L. Nehdi. 2017. “Remote sensing of concrete bridge decks using unmanned aerial vehicle infrared thermography.” Autom. Constr. 83 (Nov): 360–371. https://doi.org/10.1016/j.autcon.2017.06.024.
Omar, T., M. L. Nehdi, and T. Zayed. 2017. “Performance of NDT techniques in appraising condition of reinforced concrete bridge decks.” J. Perform. Constr. Facil. 31 (6): 04017104. https://doi.org/10.1061/(ASCE)CF.1943-5509.0001098.
Omar, T., M. L. Nehdi, and T. Zayed. 2018a. “Infrared thermography model for automated detection of delamination in RC bridge decks.” Constr. Build. Mater. 168 (Apr): 313–327. https://doi.org/10.1016/j.conbuildmat.2018.02.126.
Omar, T., M. L. Nehdi, and T. Zayed. 2018b. “Rational condition assessment of RC bridge decks subjected to corrosion-induced delamination.” J. Mater. Civ. Eng. 30 (1): 04017259. https://doi.org/10.1061/(ASCE)MT.1943-5533.0002114.
Pan, N. H., C. H. Tsai, K. Y. Chen, and J. Sung. 2020. “Enhancement of external wall decoration material for the building in safety inspection method.” J. Civ. Eng. Manage. 26 (3): 216–226. https://doi.org/10.3846/jcem.2020.11925.
Pozzer, S., Z. M. C. Pravia, E. R. Azar, and F. D. Dalla Rosa. 2020. “Statistical analysis of favorable conditions for thermographic inspection of concrete slabs.” J. Civ. Struct. Health Monit. 10 (4): 609–626. https://doi.org/10.1007/s13349-020-00405-4.
Rubio, J. J., T. Kashiwa, T. Laiteerapong, W. Deng, K. Nagai, S. Escalera, K. Nakayama, Y. Matsuo, and H. Prendinger. 2019. “Multi-class structural damage segmentation using fully convolutional networks.” Comput. Ind. 112 (Nov): 103121. https://doi.org/10.1016/j.compind.2019.08.002.
Sandler, M., A. Howard, M. Zhu, A. Zhmoginov, and L. C. Chen. 2018. “Mobilenetv2: Inverted residuals and linear bottlenecks.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 4510–4520. New York: IEEE. https://doi.org//10.1109/CVPR.2018.00474.
Simonyan, K., and A. Zisserman. 2015. “Very deep convolutional networks for large-scale image recognition.” Proc., Int. Conf. on Learning Representations, 1–14. San Diego: Cornell Univ. Library. http://arxiv.org/abs/1409.1556.
Sony, S., S. Laventure, and A. Sadhu. 2019. “A literature review of next-generation smart sensing technology in structural health monitoring.” Struct. Control Health Monit. 26 (3): e2321. https://doi.org/10.1002/stc.2321.
Stewart, M. G., X. Wang, and M. N. Nguyen. 2011. “Climate change impact and risks of concrete infrastructure deterioration.” Eng. Struct. 33 (4): 1326–1337. https://doi.org/10.1016/j.engstruct.2011.01.010.
Tran, Q. H., D. Han, C. Kang, A. Haldar, and J. Huh. 2017. “Effects of ambient temperature and relative humidity on subsurface defect detection in concrete structures by active thermal imaging.” Sensors 17 (8): 1718. https://doi.org/10.3390/s17081718.
Watase, A., R. Birgul, S. Hiasa, M. Matsumoto, K. Mitani, and F. N. Catbas. 2015. “Practical identification of favorable time windows for infrared thermography for concrete bridge evaluation.” Constr. Build. Mater. 101 (Dec): 1016–1030. https://doi.org/10.1016/j.conbuildmat.2015.10.156.
Zhao, Z. Q., P. Zheng, S. T. Xu, and X. Wu. 2019. “Object detection with deep learning: A review.” IEEE Trans. Neural Networks Learn. Syst. 30 (11): 3212–3232. https://doi.org/10.1109/TNNLS.2018.2876865.
Information & Authors
Information
Published In
Journal of Performance of Constructed Facilities
Volume 35 • Issue 1 • February 2021
Copyright
© 2020 American Society of Civil Engineers.
History
Received: Apr 28, 2020
Accepted: Aug 5, 2020
Published online: Oct 31, 2020
Published in print: Feb 1, 2021
Discussion open until: Mar 31, 2021
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.