Machine Learning Approach to Visual Bridge Inspection with Drones
Publication: Structures Congress 2022
ABSTRACT
This paper presents a machine learning approach coupled with image visibility optimization techniques to enable more efficient visual bridge inspection when relying solely on remote-controlled drones. This approach involved convolutional neural network (CNN) considered a representative machine learning algorithm and image analysis of optimized high-resolution imagery collected with drones. To evaluate the efficiency of this approach, remote-controlled drones were utilized to detect and measure the damage of two timber bridges located in Minnesota. Visual inspections on each of the bridges were initially performed in 2019 with two drones (DJI Phantom 4 and DJI Matrice 210), which resulted in a total of 66,572 extracted images. Then, the CNN was trained with a large number of extracted images containing varying damage types (i.e., cracking, weathering, and spalling) and attempted to classify the damage in an effective fashion. The visibility of the images that were classified per damage type through the CNN training was optimized by fine-tuning different properties of its images to take a measurement of damage specific to critical sections for each bridge. Included in the image properties were brightness, contrast, and sharpness. Through the analysis of extracted images from both timber bridges, the integrated CNN coupled with an image visibility optimization approach demonstrated the capability of improving the visibility of the damage and the accuracy of damage measurement.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Duque, L., Seo, J., and Wacker, J. P. (2018a). “Timber Bridge Inspection Using UAV.” Structures Congress 2018, Fort Worth, TX, 186–196.
Duque, L., Seo, J., and Wacker, J. P. (2018b). “Synthesis of Unmanned Aerial Vehicle Applications for Infrastructures.” Journal of Performance of Constructed Facilities, 32(4), 04018046.
Duque, L., Seo, J., and Wacker, J. P. (2018c). “Bridge Deterioration Quantification Protocol Using UAV.” Journal of Bridge Engineering, 23(10), 04018080.
FHWA (Federal Highway Administration). (1995). Recording and Coding Guide for the Structure Inventory and Appraisal of the Nation’s Bridges. US DOT FHWA, Washington, DC, 124.
FHWA (Federal Highway Administration). (2017). National Bridge Inventory (NBI) Data., USDOT FHWA, Washington, DC.
FHWA (Federal Highway Administration). (2012). Recording and Coding Guide for the Structure Inventory and Appraisal of the Nation’s Bridges. US DOT FHWA, Washington, DC, 124.
FHWA (Federal Highway Administration). (2018). Recording and Coding Guide for the Structure Inventory and Appraisal of the Nation’s Bridges. US DOT FHWA, Washington, DC, 124.
Fujita, Y., and Hamamoto, Y. (2011). “A robust automatic crack detection method from noisy concrete surfaces.” Machine Vision and Applications, 22(2), 245–254.
Hallermann, N., and Morgenthal, G. (2014). “Visual inspection strategies for large bridges using Unmanned Aerial Vehicles (UAV).” In Proc. of 7th IABMAS, International Conference on Bridge Maintenance, Safety and Management, 661–667.
Jahanshahi, M. R., Masri, S. F., Padgett, C. W., and Sukhatme, G. S. (2013). “An innovative methodology for detection and quantification of cracks through incorporation of depth perception.” Machine Vision and Applications, 24(2), 227–241.
Jeong, E., Seo, J., and Wacker, J. P. (2020a). “New Bridge Inspection Approach with Joint UAV and DIC System.” Structures Congress 2020, St. Louis, MO, 349–359.
Jeong, E., Seo, J., and Wacker, J. P. (2020b). “Non-Contact Bridge Inspection and Damage Quantification Approach using UAS.” US-Korea Conference (UKC) 2020 – Virtual Event, Korean-American Scientists and Engineers Association (KSEA).
Jeong, E., Seo, J., and Wacker, J. P. (2020c). “Literature Review and Technical Survey on Bridge Inspection Using Unmanned Aerial Vehicles.” Journal of Performance of Constructed Facilities, 34(6), 04020113.
Jeong, E., Seo, J., and Wacker, J. P. (2021a). UAV-Aided Bridge Inspection Protocol through Machine Learning with Improved Visibility Images.
Jeong, E., Seo, J., and Wacker, J. P. (2021b). “Grayscale Drone Inspection Image Enhancement Framework for Advanced Bridge Defect Measurement.” Transportation Research Record, SAGE Publications Inc, 2675(8), 603–612.
Khaloo, A., Lattanzi, D., Cunningham, K., Dell’Andrea, R., and Riley, M. (2018). “Unmanned aerial vehicle inspection of the Placer River Trail Bridge through image-based 3D modelling.” Structure and Infrastructure Engineering, 14(1), 124–136.
Lee, B. J., Shin, D. H., Seo, J. W., Jung, J. D., and Lee, J. Y. (2011). “Intelligent bridge inspection using remote controlled robot and image processing technique.” International Symposium on Automation and Robotics in Construction (ISARC), Seoul, Korea, 1426–1431.
MATLAB & Simulink. (2021). “ResNet-50 convolutional neural network - MATLAB resnet50.” <https://www.mathworks.com/help/deeplearning/ref/resnet50.html;jsessionid=43a5026dc71f385e628fc595ade6>(Jul. 26, 2021).
Oh, J.-K., Jang, G., Oh, S., Lee, J. H., Yi, B.-J., Moon, Y. S., Lee, J. S., and Choi, Y. (2009). “Bridge inspection robot system with machine vision.” Automation in Construction, 18(7), 929–941.
Reagan, D., Sabato, A., and Niezrecki, C. (2017a). “Feasibility of using digital image correlation for unmanned aerial vehicle structural health monitoring of bridges.” Structural Health Monitoring, 17(5), 1056–1072.
Reagan, D., Sabato, A., Niezrecki, C., Reagan, D., Sabato, A., and Niezrecki, C. (2017b). “Unmanned aerial vehicle acquisition of three-dimensional digital image correlation measurements for structural health monitoring of bridges.” T.-Y. Yu, ed., Portland, Oregon, United States, 1016909.
Seo, J., Duque, L., and Wacker, J. P. (2018a). “Drone-enabled bridge inspection methodology and application.” Automation in Construction, 94, 112–126.
Seo, J., Jeong, E., and Wacker, J. P. (2020). “Drone-Aided Bridge Inspection Coupled with Image Quality Improvement Processing Technique.” NDE/NDT Structural Materials Technology for Highways and Bridges 2020, Online Only, OH.
Seo, J., Jeong, E., and Wacker, J. P. (2021a). “Visual Bridge Damage Measurement Using Drone-Captured Image Quality Optimization.” European Workshop on Structural Health Monitoring, P. Rizzo and A. Milazzo, eds., Springer International Publishing, Cham, 503–513.
Seo, J., Jeong, E., and Wacker, J. P. (2021b). “Advanced Defect Identification and Quantification Using Grayscale UAS Inspection Images.” US-Korea Conference (UKC) 2021, Korean-American Scientists and Engineers Association (KSEA), Los Angeles area, CA.
Seo, J., Jeong, E., and Wacker, J. P. (2021c). “Visible Damage Examination of Concrete Bridge Decking using UAS.” ACI Virtual Concrete Convention, American Concrete Institue (ACI).
Seo, J., Wacker, J. P., and Duque, L. (2018b). Evaluating the use of drones for timber bridge inspection., US Department of Agriculture, Forest Service, Forest Products Laboratory, Madison, WI, 145.
UAV ForecastTM. (2019). “UAV Forecast.” <http://www.UAVForecast.com>(Jul. 22, 2019).
Ward, P., Manamperi, P., Brooks, P., Mann, P., Kaluarachchi, W., Matkovic, L., Paul, G., Yang, C.-H., Quin, P., Pagano, D., Liu, D., Waldron, K., and Dissanayake, G. (2014). “Climbing robot for steel bridge inspection: Design challenges.” Austroads Bridge Conference, ARRB Group.
Zink, J., and Lovelace, B. (2015). Unmanned Aerial Vehicle Bridge Inspection Demonstration Project., Minnesota Department of Transportation, Oakdale, MN, 214.
Information & Authors
Information
Published In
Structures Congress 2022
Pages: 160 - 169
History
Published online: Apr 18, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.