Real-Time AI-Based Bridge Inspection Using Mixed Reality Platform
Publication: Structures Congress 2023
ABSTRACT
Conventional methods in bridge inspection are labor extensive and highly subjective. This study introduces a novel approach with the use of real-time machine learning and the mixed reality platform to assist the inspector in localizing and quantifying concrete’s surface defects. Two separate deep learning models were selected for real-time detection and segmentation of the defects. These models were chosen from several available deep learning models based on their accuracy, inference speed, and memory size. For defect localization, Yolov5s showed the most promising results compared to several other CNN architectures including EfficientDet-d0. For the defect quantification model, 12 different architectures were trained and compared. UNet with EfficientNet-b0 backbone was found to be the best performing model in terms of inference speed and accuracy. The selected models were then quantized and deployed in a mixed reality platform and image tracking libraries were configured in the platform environment. An accurate distance estimation was accomplished using spatial meshing. Lastly, a methodology for condition assessment of concrete defects using the mixed reality system is discussed. The proposed approach eliminates the subjectivity of human inspection and reduces labor time. It also guarantees human-verified results and eliminates the need for post-processing.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Abdel-Qader, I., O. Abudayyeh, and M. E. Kelly. 2003. “Analysis of Edge-Detection Techniques for Crack Identification in Bridges.” J. Comput. Civ. Eng., 17 (4): 255–263.
Adhikari, R. S., O. Moselhi, and A. Bagchi. 2014. “Image-based retrieval of concrete crack properties for bridge inspection.” Autom. Constr., 39: 180–194. Elsevier B.V.
Al-Sabbag, Z. A., C. M. Yeum, and S. Narasimhan. 2022. “Distributed Collaboration in Infrastructure Assessment through Mixed and Virtual Reality - a.” Transform. Constr. with Real. Capture Technol..
Bae, H., M. Golparvar-Fard, and J. White. 2013. “High-precision vision-based mobile augmented reality system for context-aware architectural, engineering, construction and facility management (AEC/FM) applications.” Vis. Eng., 1 (1): 3.
Behzadan, A. H., and V. R. Kamat. 2007. “Georeferenced Registration of Construction Graphics in Mobile Outdoor Augmented Reality.” J. Comput. Civ. Eng., 21 (4): 247–258.
Bianchi, E., A. L. Abbott, P. Tokekar, and M. Hebdon. 2021. “COCO-Bridge: Structural Detail Data Set for Bridge Inspections.” J. Comput. Civ. Eng., 35 (3): 04021003. American Society of Civil Engineers.
Brito, C., N. Alves, L. Magalhães, and M. Guevara. 2019. “BIM Mixed Reality Tool for the Inspection of Heritage Buildings.”.
Dang, N. S., and C. S. Shim. 2020. “BIM-based innovative bridge maintenance system using augmented reality technology.” Lect. Notes Civ. Eng., 54: 1217–1222. Springer.
Dorafshan, S., R. J. Thomas, and M. Maguire. 2018. “SDNET2018: An annotated image dataset for non-contact concrete crack detection using deep convolutional neural networks.” Data Br., 21: 1664–1668. Elsevier Inc.
Eslami, E., and H. B. Yun. 2021. “Attention-Based Multi-Scale Convolutional Neural Network (A+MCNN) for Multi-Class Classification in Road Images.” Sensors, 21 (15): 5137. Multidisciplinary Digital Publishing Institute.
Ioannis, B. 2017. “Mixed reality constructs a new frontier for maintaining the built environment.” Proc. Inst. Civ. Eng. - Civ. Eng., 170 (2): 53.
Jaccard, P. 1912. “The Distribution of the Flora in the Alpine Zone.” New Phytol..
Kamat, V. R., and S. El-Tawil. 2007. “Evaluation of Augmented Reality for Rapid Assessment of Earthquake-Induced Building Damage.” J. Comput. Civ. Eng., 21 (5): 303–310.
Karaaslan, E., U. Bagci, and F. N. Catbas. 2021. “Attention-guided analysis of infrastructure damage with semi-supervised deep learning.” Autom. Constr., 125 (February): 103634. Elsevier B.V.
Karaaslan, E., B. Sen, T. Ercan, H. Laman, and J. Pol. 2020. “Artificial Intelligence Embedded On-Board Machine Vision System to Support Vehicle to Infrastructure.” Transp. Res. Rec.
Karaaslan, E., M. Zakaria, and F. N. Catbas. 2022. “Mixed reality-assisted smart bridge inspection for future smart cities.” Rise Smart Cities, 261–280. Butterworth-Heinemann.
Koch, C., K. Georgieva, V. Kasireddy, B. Akinci, and P. Fieguth. 2015. “A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure.” Adv. Eng. Informatics, 29 (2): 196–210. Elsevier Ltd.
LaLonde, R., and U. Bagci. 2018. “Capsules for Object Segmentation.”
LaViola, J., E. Kruijff, D. Bowman, and I. Poupyrev. 2017. 3D User Interfaces: Theory and Practice, Second Edition. Recherche.
Maharjan, D., M. Agüero, D. Mascarenas, R. Fierro, and F. Moreu. 2020. “Enabling human–infrastructure interfaces for inspection using augmented reality.” Structural Health Monitoring, 20 (4): 1980–1996. SAGE PublicationsSage UK: London, England.
Mansoor, A., U. Bagci, B. Foster, Z. Xu, G. Z. Papadakis, L. R. Folio, J. K. Udupa, and D. J. Mollura. 2015. “Segmentation and Image Analysis of Abnormal Lungs at CT: Current Approaches, Challenges, and Future Trends.” Radiographics, 35 (4): 1056. Radiological Society of North America.
Microsoft. 2017. “The leader in Mixed Reality Technology | HoloLens.” Microsoft.
Moreu, F., B. Bleck, S. Vemuganti, D. Rogers, and D. Mascarenas. 2017. “Augmented reality tools for enhanced structural inspection.” Struct. Heal. Monit. 2017 Real-Time Mater. State Aware. Data-Driven Saf. Assur. - Proc. 11th Int. Work. Struct. Heal. Monit. IWSHM 2017, 3124–3130.
Mundt, M., S. Majumder, S. Murali, P. Panetsos, and V. Ramesh. 2019. “CODEBRIM: COncrete DEfect BRidge IMage Dataset.”.
Nguyen, D.-C., R. Jin, C.-H. Jeon, and C.-S. Shim. 2022. “BIM-based mixed-reality application for bridge inspection and maintenance.” 22 (3): 487–503.
Ren, Y., J. Huang, Z. Hong, W. Lu, J. Yin, L. Zou, and X. Shen. 2020. “Image-based concrete crack detection in tunnels using deep fully convolutional networks.” Constr. Build. Mater., 234: 117367. Elsevier Ltd.
Yokoyama, S., and T. Matsumoto. 2017. “Development of an Automatic Detector of Cracks in Concrete Using Machine Learning.” Procedia Eng., 171: 1250–1255.
Zakaria, M., E. Karaaslan, and F. N. Catbas. 2022. “Advanced Bridge Visual Inspection Using Real-time Machine Learning in Edge Devices.” Adv. Bridg. Eng.
Zhang, C., C. C. Chang, and M. Jamshidi. 2021. “Simultaneous pixel-level concrete defect detection and grouping using a fully convolutional model.” Struct. Heal. Monit., 20 (4): 2199–2215.
Zhang, L., J. Shen, and B. Zhu. 2020a. “A research on an improved Unet-based concrete crack detection algorithm.” Struct. Heal. Monit., (29).
Zhang, Q., K. Barri, S. K. Babanajad, and A. H. Alavi. 2020b. “Real-Time Detection of Cracks on Concrete Bridge Decks Using Deep Learning in the Frequency Domain.” Engineering. Elsevier BV.
Information & Authors
Information
Published In
Structures Congress 2023
Pages: 120 - 131
History
Published online: May 1, 2023
ASCE Technical Topics:
- Architectural engineering
- Architecture
- Artificial intelligence and machine learning
- Bridge engineering
- Bridges
- Business management
- Computer programming
- Computing in civil engineering
- Concrete
- Construction engineering
- Construction management
- Defects and imperfections
- Employment
- Engineering fundamentals
- Engineering materials (by type)
- Inspection
- Labor
- Materials characterization
- Materials engineering
- Model accuracy
- Models (by type)
- Neural networks
- Personnel management
- Practice and Profession
- Structural engineering
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.