Automated Rust-Defect Detection of a Steel Bridge Using Aerial Multispectral Imagery
Publication: Journal of Infrastructure Systems
Volume 25, Issue 2
Abstract
Computer vision methods have the potential to detect rust defects in steel components of bridges. However, direct use of images collected by aerial means to identify such defects is currently difficult because of obstructions caused by other objects in the image field of view. In this context, an automated rust-defect-determination method that leverages aerial imagery, including both visible and infrared images, is presented in this investigation. The proposed method consists of three steps. The first step deals with image registration for which a binary information method is proposed to match the infrared images to their visible counterparts. In the second step, bridge components are retrieved from the captured images via automated segmentation obtained by fusion of visible and infrared images. Finally, rusted regions are identified in YCbCr colorspace, and a rust percentage is calculated. Experimental results obtained by aerial images collected on a real operating structure demonstrate that the proposed methodology can directly use the original captured images and can be successfully applied to real-world scenarios.
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgments
Yundong Li’s research work was supported by Beijing Natural Science Foundation (4182020). The authors would like to acknowledge Dr. Fuad Khan and Dr. Andrew Ellenberg for their help in preparing data collection. This research was partially supported by the National Science Foundation (1538389).
References
Acosta, M. R. G., J. C. V. Díaz, and N. S. Castro. 2014. “An innovative image-processing model for rust detection using Perlin noise to simulate oxide textures.” Corros. Sci. 88 (11): 141–151. https://doi.org/10.1016/j.corsci.2014.07.027.
Agbelie, R. D. K., S. Labi, J. Fricker, Y. Qiao, Z. B. Zhang, and K. C. Sinha. 2017. “Life cycle decision framework for steel bridge painting.” J. Bridge Eng. 22 (11): 06017004. https://doi.org/10.1061/(ASCE)BE.1943-5592.0001140.
Almadhoun, R., T. Taha, J. D. Seneviratne, and G. W. Cai. 2016. “A survey on inspecting structures using robotic systems.” Int. J. Adv. Rob. Syst. 13 (6): 1–8. https://doi.org/10.1177/1729881416663664.
ASTM. 2012. Standard practice for evaluating degreed of rusting on painted steel surface. ASTM D610-08. West Conshohocken, PA: ASTM.
Chen, P. H., Y. C. Chang, and L. M. Chang. 2009. “Automated bridge coating defect recognition using adaptive ellipse approach.” Autom. Constr. 18 (5): 632–643. https://doi.org/10.1016/j.autcon.2008.12.007.
Chen, P. H., Y. C. Chang, L. M. Chang, and P. C. Doerschuk. 2002. “Application of multiresolution pattern classification to steel bridge coating assessment.” J. Comput. Civ. Eng. 16 (4): 244–251. https://doi.org/10.1061/(ASCE)0887-3801(2002)16:4(244).
Chen, P. H., H. K. Shen, C. Y. Lei, and L. M. Chang. 2011. “Fourier-transform-based method for automated steel bridge coating defect recognition.” Procedia Eng. 14 (3): 470–476. https://doi.org/10.1016/j.proeng.2011.07.058.
Chen, P. H., H. K. Shen, C. Y. Lei, and L. M. Chang. 2012. “Support-vector-machine-based method for automated steel bridge rust assessment.” Autom. Constr. 23: 9–19. https://doi.org/10.1016/j.autcon.2011.12.001.
Choi, K. Y., and S. S. Kim. 2005. “Morphological analysis and classification of types of surface corrosion damage by digital image processing.” Corros. Sci. 47 (1): 1–15. https://doi.org/10.1016/j.corsci.2004.05.007.
Ellenberg, A., L. Branco, A. Krick, I. Bartoli, and A. Kontsos. 2015. “Use of unmanned aerial vehicle for quantitative infrastructure evaluation.” J. Infrastruct. Syst. 21 (3): 04014054. https://doi.org/10.1061/(ASCE)IS.1943-555X.0000246.
Ellenberg, A., A. Kontsos, I. Bartoli, and A. Pradhan. 2014. “Masonry crack detection application of an unmanned aerial vehicle.” In Proc., Int. Conf. on Computing in Civil and Building Engineering 2014. Reston, VA: ASCE.
Ellenberg, A., A. Kontsos, F. Moon, and I. Bartoli. 2016. “Bridge related damage quantification using unmanned aerial vehicle imagery.” Struct. Control Health Monit. 23 (9): 1168–1179. https://doi.org/10.1002/stc.1831.
Ghanta, S., T. Karp, and S. W. Lee. 2011. “Wavelet domain detection of rust steel bridge images.” In Proc., IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP) 2011. Piscataway, NJ: IEEE.
Ham, Y., K. K. Han, J. J. Lin, and M. Golparvar-Fard. 2016. “Visual monitoring of civil infrastructure systems via camera-equipped unmanned aerial vehicles (UAV): A review of related works.” Visualization Eng. 4 (1): 1. https://doi.org/10.1186/s40327-015-0029-z.
Hwang, N., H. Son, C. Kim, and C. Kim. 2013. “Rust surface area determination of steel bridge component for robotic grit-blast machine.” In Proc., 30th Int. Symp. on Automation and Robotics in Construction and Mining. Vilnius, Lithuania: Vilnius Gediminas Technical Univ.
Jahanshahi, M. R., J. S. Kelly, S. F. Masri, and G. S. Sukhatme. 2009. “A survey and evaluation of promising approaches for automatic image-based defect detection of bridge structures.” Struct. Infrastruct. Eng. 5 (6): 455–486. https://doi.org/10.1080/15732470801945930.
Khan, F., A. Ellenberg, M. Mazzotti, A. Kontsos, F. Moo, A. Pradhan, and I. Bartoli. 2015. “Investigation on bridge assessment using unmanned aerial systems.” In Proc., Structures Congress 2015. Reston, VA: ASCE.
Kim, H., J. Lee, E. Ahn, S. Cho, M. Shin, and S. H. Sim. 2017. “Concrete crack identification using a UAV incorporating hybrid image processing.” Sensors 17 (9): C1–C1. https://doi.org/10.1109/JSEN.2017.2685478.
Lattanzi, D., and G. Miller. 2017. “Review of robotic infrastructure inspection systems.” J. Infrastruct. Syst. 23 (3): 04017004. https://doi.org/10.1061/(ASCE)IS.1943-555X.0000353.
Lee, S. W., S. Ghanta, and T. Karp. 2012. “Comparative study o retrospective methods to reduce non-uniform illumination effects to bridge coating.” Autom. Constr. 22: 537–544. https://doi.org/10.1016/j.autcon.2011.11.014.
Li, H. G., W. R. Ding, X. B. Cao, and C. L. Liu. 2017a. “Image registration and fusion of visible and infrared integrated camera for medium-altitude unmanned aerial vehicle remote sensing.” Remote Sens. 9 (5): 441. https://doi.org/10.3390/rs9050441.
Li, Y. D., W. G. Zhao, and J. H. Pan. 2017b. “Deformable patterned fabric defect detection with Fisher criterion-based deep learning.” IEEE Trans. Autom. Sci. Eng. 14 (2): 1256–1264. https://doi.org/10.1109/TASE.2016.2520955.
Liao, K. W., and Y. T. Lee. 2016. “Detection of rust defects on steel bridge coatings via digital image recognition.” Autom. Constr. 71: 294–306. https://doi.org/10.1016/j.autcon.2016.08.008.
Liu, F., and S. Seipel. 2015. “Infrared-visible image registration for augmented reality-based thermographic building diagnostics.” Visualization Eng. 3 (1): 16. https://doi.org/10.1186/s40327-015-0028-0.
Mader, D., R. Blaskow, P. Westfeld, and C. Weller. 2016. “Potential of UAV-based laser scanner and multispectral camera data in building inspection.” In The international archives of the photogrammetry, remote sensing and spatial information sciences. Prague, Czech Republic: International Society of Photogrammetry and Remote Sensing.
Omar, T., and M. L. Nehdi. 2017. “Remote sensing of concrete bridge using unmanned aerial vehicle infrared thermography.” Autom. Constr. 83: 360–371. https://doi.org/10.1016/j.autcon.2017.06.024.
Otsu, N. 1979. “A threshold selection method from gray-level histograms.” IEEE Trans. Syst. Man Cybern. 9 (1): 62–66. https://doi.org/10.1109/TSMC.1979.4310076.
Phung, M. D., T. H. Dinh, V. T. Hoang, and Q. P. Ha. 2017. “Automatic crack detection in build infrastructure using unmanned aerial vehicles.” Preprint, submitted July 31, 2017. http://arxiv.org/abs/1707.09715v1.
Shen, H. K., P. H. Chen, and L. M. Chang. 2013. “Automated steel bridge coating rust defect recognition method based on color and texture feature.” Autom. Constr. 31: 338–356. https://doi.org/10.1016/j.autcon.2012.11.003.
Shih, P., and C. Liu. 2005. “Comparative assessment of content-based face image retrieval in different color spaces.” Int. J. Pattern Recognit. Artif. Intell. 19 (7): 873–893. https://doi.org/10.1142/S0218001405004381.
Son, H. J., N. Y. Hwang, C. M. Kin, and C. W. Kim. 2014. “Rapid and automated determination of rusted surface areas of a steel bridge for robotic maintenance systems.” Autom. Constr. 42: 13–24. https://doi.org/10.1016/j.autcon.2014.02.016.
Wu, F. H., B. J. Wang, X. Yi, M. Li, J. Y. Hao, H. L. Qin, and H. X. Zhou. 2015. “Visible and infrared image registration based on visual salient features.” J. Electron. Imaging 24 (5): 053017. https://doi.org/10.1117/1.JEI.24.5.053017.
Information & Authors
Information
Published In
Journal of Infrastructure Systems
Volume 25 • Issue 2 • June 2019
Copyright
©2019 American Society of Civil Engineers.
History
Received: Feb 15, 2018
Accepted: Nov 9, 2018
Published online: Mar 29, 2019
Published in print: Jun 1, 2019
Discussion open until: Aug 29, 2019
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.