Optimized Lightweight Edge Computing Platform for UAV-Assisted Detection of Concrete Deterioration beneath Bridge Decks
Publication: Journal of Computing in Civil Engineering
Volume 39, Issue 1
Abstract
This study introduces a transformative artificial intelligence of things (AIoT) framework that advances bridge maintenance by incorporating advanced inspection techniques. A central innovation is the Pilgrimage Walk Optimization (PWO)-Lite algorithm, which fine-tunes the hyperparameters of the You Only Look Once (YOLO)v7-tiny deep learning model. This model, integrated with the Deep Simple Online and Realtime Tracking (DeepSORT) algorithm, enables real-time detection and significantly enhances the system’s ability to detect deteriorations in concrete beneath bridge decks swiftly and accurately. The PWO-Lite algorithm draws inspiration from the traditional Matsu pilgrimage, an important Taiwanese folk religious event. It reflects this influence in its search behavior, miming devotees’ gathering and movement patterns. This unique approach to algorithmic design incorporates cultural customs into computational strategies. An embedded system has been configured to efficiently process visual data from unmanned aerial vehicles (UAVs), providing actionable insights directly at the inspection site. This configuration reduces the reliance on heavy computational equipment and complex setups, streamlining bridge inspections and minimizing dependence on extensive infrastructure. The practical integration of this technology into UAVs allows engineers and field professionals to obtain precise, real-time data, enhancing maintenance planning and resource management. The broader implications of this research include the potential to significantly improve standard practices in infrastructure maintenance, offering a scalable solution that could revolutionize the field. This study bridges the gap between traditional AI applications and civil engineering. It introduces a culturally inspired optimization technique to structural health monitoring, benefiting both theoretical and practical aspects of infrastructure maintenance.
Practical Applications
This research introduces a cutting-edge system that combines artificial intelligence with advanced drone-mounted cameras to inspect the condition of concrete beneath bridge decks more efficiently. Utilizing the newly developed Pilgrimage Walk Optimization (PWO)-Lite algorithm, this system quickly identifies and analyzes deterioration, such as cracks or spalling, without requiring large-scale computer systems or intensive manual labor. By integrating this technology into drones, field experts and engineers can swiftly transform images of bridge deterioration into actionable insights, expediting assessments and reducing maintenance costs. This approach enhances maintenance scheduling and budget allocation and significantly shortens the time required for bridge inspections. Our system enables real-time data collection and analysis, which is particularly beneficial in remote or difficult-to-access areas. These innovations provide practical, scalable solutions for infrastructure management, significantly improving the safety and longevity of bridge structures through timely maintenance and precise damage assessment.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or codes that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
The authors would like to express their gratitude to the National Science and Technology Council, Taiwan, and UTRUST Co., Ltd. for financially supporting this research under grant NSTC 110-2221-E-011-080-MY3, NSTC 113-2811-E-011-002, and NTUST-UTRUST 111A09061.
References
Bewley, A., Z. Ge, L. Ott, F. Ramos, and B. Upcroft. 2016. “Simple online and realtime tracking.” In Proc., 2016 IEEE Int. Conf. on Image Processing (ICIP), 3464–3468. New York: IEEE. https://doi.org/10.1109/ICIP.2016.7533003.
Bin Zuraimi, M. A., and F. H. K. Zaman. 2021. “Vehicle detection and tracking using YOLO and DeepSORT.” In Proc., 11th IEEE Symp. on Computer Applications and Industrial Electronics (ISCAIE), 23–29. New York: IEEE. https://doi.org/10.1109/iscaie51753.2021.9431784.
Bochkovskiy, A., C. Y. Wang, and H. Y. M. Liao. 2020. “YOLOv4: Optimal speed and accuracy of object detection.” Preprint, submitted April 23, 2020. http://arxiv.org/abs/2004.10934.
Bura, H., N. Lin, N. Kumar, S. Malekar, S. Nagaraj, and K. Liu. 2018. “An edge based smart parking solution using camera networks and deep learning.” In Proc., 2018 IEEE Int. Conf. on Cognitive Computing (ICCC), 17–24. New York: IEEE. https://doi.org/10.1109/ICCC.2018.00010.
Chen, C., B. Liu, S. H. Wan, P. Qiao, and Q. Q. Pei. 2021. “An edge traffic flow detection scheme based on deep learning in an intelligent transportation system.” IEEE Trans. Intell. Transp. Syst. 22 (3): 1840–1852. https://doi.org/10.1109/TITS.2020.3025687.
Chen, F.-C., A. Subedi, M. R. Jahanshahi, D. R. Johnson, and E. J. Delp. 2022. “Deep learning–based building attribute estimation from Google street view images for flood risk assessment using feature fusion and task relation encoding.” J. Comput. Civ. Eng. 36 (6): 04022031. https://doi.org/10.1061/(ASCE)CP.1943-5487.0001025.
Chou, J.-S., and C.-Y. Liu. 2023. “Pilgrimage walk optimization: Folk culture-inspired algorithm for identification of bridge deterioration.” Autom. Constr. 155 (Nov): 105055. https://doi.org/10.1016/j.autcon.2023.105055.
CyberLink. 2023. “PowerDirector.” Accessed April 23, 2023. https://tw.cyberlink.com/products/powerdirector-video-editing-software/overview_zh_TW.html.
Dai, J., H. Qi, Y. Xiong, Y. Li, G. Zhang, H. Hu, and Y. Wei. 2017. “Deformable convolutional networks.” In Proc., IEEE Int. Conf. on Computer Vision, 764–773. New York: IEEE. https://doi.org/10.1109/ICCV.2017.89.
Du, F., S. Jiao, and K. Chu. 2022. “Application research of bridge damage detection based on the improved lightweight convolutional neural network model.” Appl. Sci. 12 (12): 6225. https://doi.org/10.3390/app12126225.
Feng, H., G. Mu, S. Zhong, P. Zhang, and T. Yuan. 2021. “Benchmark analysis of YOLO performance on edge intelligence devices.” In Proc., 2021 Cross Strait Radio Science and Wireless Technology Conf. (CSRSWTC), 319–321. New York: IEEE. https://doi.org/10.1109/CSRSWTC52801.2021.9631594.
Fujita, Y., K. Shimada, M. Ichihara, and Y. Hamamoto. 2017. “A method based on machine learning using hand-crafted features for crack detection from asphalt pavement surface images.” In Proc., Int. Conf. on Quality Control by Artificial Vision 2017, 103380I. Washington, DC: SPIE. https://doi.org/10.1117/12.2264075.
Gai, Y. Q., W. Y. He, and Z. L. Zhou. 2021. “Pedestrian target tracking based on DeepSORT with YOLOv5.” In Proc., 2nd Int. Conf. on Computer Engineering and Intelligent Control (ICCEIC), 1–5. New York: IEEE. https://doi.org/10.1109/icceic54227.2021.00008.
Ge, Y. H., S. Lin, Y. H. Zhang, Z. L. Li, H. T. Cheng, J. Dong, S. S. Shao, J. Zhang, X. Y. Qi, and Z. D. Wu. 2022. “Tracking and counting of tomato at different growth period using an improving YOLO-Deepsort network for inspection robot.” Machines 10 (6): 489. https://doi.org/10.3390/machines10060489.
Glenn Jocher, A. S., et al. 2021. “ultralytics/yolov5: v5.0—YOLOv5-P6 1280 models.” Accessed March 14, 2023. https://github.com/ultralytics/yolov5.
He, K., X. Zhang, S. Ren, and J. Sun. 2015. “Spatial pyramid pooling in deep convolutional networks for visual recognition.” IEEE Trans. Pattern Anal. Mach. Intell. 37 (9): 1904–1916. https://doi.org/10.1109/TPAMI.2015.2389824.
Kao, S.-P., Y.-C. Chang, and F.-L. Wang. 2023. “Combining the YOLOv4 deep learning model with UAV imagery processing technology in the extraction and quantization of cracks in bridges.” Sensors 23 (5): 2572. https://doi.org/10.3390/s23052572.
Karaboga, D., and B. Akay. 2009. “A comparative study of artificial bee colony algorithm.” Appl. Math. Comput. 214 (1): 108–132. https://doi.org/10.1016/j.amc.2009.03.090.
Karaboga, D., and B. Basturk. 2007. “A powerful and efficient algorithm for numerical function optimization: Artificial bee colony (ABC) algorithm.” J. Global Optim. 39 (3): 459–471. https://doi.org/10.1007/s10898-007-9149-x.
Karaman, A., D. Karaboga, I. Pacal, B. Akay, A. Basturk, U. Nalbantoglu, S. Coskun, and O. Sahin. 2023. “Hyper-parameter optimization of deep learning architectures using artificial bee colony (ABC) algorithm for high performance real-time automatic colorectal cancer (CRC) polyp detection.” Appl. Intell. 53 (12): 15603–15620. https://doi.org/10.1007/s10489-022-04299-1.
Kennedy, J., and R. Eberhart. 1995. “Particle swarm optimization.” In Vol. 4 of Proc., ICNN’95—Int. Conf. on Neural Networks, 1942–1948. New York: IEEE. https://doi.org/10.1109/ICNN.1995.488968.
Lee, J. G., J. Hwang, S. Chi, and J. Seo. 2022. “Synthetic image dataset development for vision-based construction equipment detection.” J. Comput. Civ. Eng. 36 (5): 04022020. https://doi.org/10.1061/(ASCE)CP.1943-5487.0001035.
Li, C., et al. 2022. “YOLOv6: A single-stage object detection framework for industrial applications.” Preprint, submitted September 7, 2022. http://arxiv.org/abs/2209.02976.
Lin, T. Y., P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie. 2017. “Feature pyramid networks for object detection.” In Proc., 2017 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 936–944. New York: IEEE. https://doi.org/10.1109/CVPR.2017.106.
Lins, R. G., S. N. Givigi, A. D. M. Freitas, and A. Beaulieu. 2018. “Autonomous robot system for inspection of defects in civil infrastructures.” IEEE Syst. J. 12 (2): 1414–1422. https://doi.org/10.1109/JSYST.2016.2611244.
Mattiev, J., J. Sajovic, G. Drevensek, and P. Rogelj. 2023. “Assessment of model accuracy in eyes open and closed EEG data: Effect of data pre-processing and validation methods.” Bioengineering 10 (1): 21. https://doi.org/10.3390/bioengineering10010042.
Mirjalili, S., and A. Lewis. 2016. “The whale optimization algorithm.” Adv. Eng. Software 95 (May): 51–67. https://doi.org/10.1016/j.advengsoft.2016.01.008.
Mittal, S. 2019. “A survey on optimized implementation of deep learning models on the NVIDIA Jetson platform.” J. Syst. Archit. 97 (Aug): 428–442. https://doi.org/10.1016/j.sysarc.2019.01.011.
Mohan, S., O. Shoghli, A. Burde, and H. Tabkhi. 2021. “Low-power drone-mountable real-time artificial intelligence framework for road asset classification.” Transp. Res. Rec. 2675 (1): 39–48. https://doi.org/10.1177/0361198120965170.
Naseri, R. A. S., A. Kurnaz, and H. M. Farhan. 2023. “Optimized face detector-based intelligent face mask detection model in IoT using deep learning approach.” Appl. Soft Comput. 134 (Feb): 109933. https://doi.org/10.1016/j.asoc.2022.109933.
Poojitha, S. N., and V. Jothiprakash. 2022. “Hybrid differential evolution and krill herd algorithm for the optimal design of water distribution networks.” J. Comput. Civ. Eng. 36 (1): 04021032. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000999.
Prasanna, S., and M. El-Sharkawy. 2023. “Improving mean average precision (mAP) of camera and radar fusion network for object detection using radar augmentation.” In Vol. 465 of Proc., 7th Int. Congress of Information and Communication Technology (ICICT), 51–60. Singapore: Springer Nature. https://doi.org/10.1007/978-981-19-2397-5_6.
Rashedi, E., H. Nezamabadi-pour, and S. Saryazdi. 2009. “GSA: A gravitational search algorithm.” Inf. Sci. 179 (13): 2232–2248. https://doi.org/10.1016/j.ins.2009.03.004.
Redmon, J., S. Divvala, R. Girshick, and A. Farhadi. 2016. “You only look once: Unified, real-time object detection.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). New York: IEEE. https://doi.org/10.1109/CVPR.2016.91.
Redmon, J., and A. Farhadi. 2019. “YOLOv3: An incremental improvement.” Preprint, submitted April 8, 2018. http://arxiv.org/abs/1804.02767.
Sreekumar, U. K., R. Devaraj, Q. Li, and K. Liu. 2018. “TPCAM: Real-time traffic pattern collection and analysis model based on deep learning.” In Proc., 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI). New York: IEEE. https://doi.org/10.1109/UIC-ATC.2017.8397674.
Storn, R., and K. Price. 1997. “Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces.” J. Global Optim. 11 (4): 341–359. https://doi.org/10.1023/A:1008202821328.
Sun, P., G. Draughon, R. Hou, and J. P. Lynch. 2022. “Automated human use mapping of social infrastructure by deep learning methods applied to smart city camera systems.” J. Comput. Civ. Eng. 36 (4): 04022011. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000998.
Teng, S., Z. C. Liu, and X. D. Li. 2022. “Improved YOLOv3-based bridge surface defect detection by combining high- and low-resolution feature images.” Buildings 12 (8): 18. https://doi.org/10.3390/buildings12081225.
Tuan, N. M., Y. J. Kim, J. Y. Lee, and S. Chin. 2022. “Automatic stereo vision-based inspection system for particle shape analysis of coarse aggregates.” J. Comput. Civ. Eng. 36 (2): 12. https://doi.org/10.1061/(ASCE)CP.1943-5487.0001005.
Vanholder, H. 2016. “Efficient inference with TensorRT. In Vol. 1 of Proc., GPU Technology Conf., 2. Santa Clara, CA: NVIDIA. https://on-demand.gputechconf.com/gtc-eu/2017/presentation/23425-han-vanholder-efficient-inference-with-tensorrt.pdf.
Wang, C. Y., H. Y. M. Liao, Y. H. Wu, P. Y. Chen, J. W. Hsieh, and I. H. Yeh. 2020. “CSPNet: A new backbone that can enhance learning capability of CNN.” In Proc., 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), 1571–1580. New York: IEEE. https://doi.org/10.1109/CVPRW50498.2020.00203.
Wang, C.-Y., A. Bochkovskiy, and H.-Y. M. Liao. 2022. “YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors.” Preprint, submitted July 6, 2022. http://arxiv.org/abs/2207.02696.
Wang, D. Y., M. X. Zhang, D. J. Sheng, and W. M. Chen. 2023. “Bolt positioning detection based on improved YOLOv5 for bridge structural health monitoring.” Sensors 23 (1): 496. https://doi.org/10.3390/s23010496.
Wojke, N., A. Bewley, and D. Paulus. 2017. “Simple online and real-time tracking with a deep association metric.” In Proc., 2017 IEEE Int. Conf. on Image Processing (ICIP), 3645–3649. New York: IEEE. https://doi.org/10.1109/ICIP.2017.8296962.
Xiao, X., M. Yan, S. Basodi, C. Ji, and Y. Pan. 2020. “Efficient hyperparameter optimization in deep learning using a variable length genetic algorithm.” Preprint, submitted June 23, 2020. http://arxiv.org/abs/2006.12703.
Xu, H., M. T. Guo, N. Nedjah, J. D. Zhang, and P. Li. 2022. “Vehicle and pedestrian detection algorithm based on lightweight YOLOv3-promote and semi-precision acceleration.” IEEE Trans. Intell. Transp. Syst. 23 (10): 19760–19771. https://doi.org/10.1109/TITS.2021.3137253.
Yang, L., and H. Cai. 2023. “Cost-efficient image semantic segmentation for indoor scene understanding using weakly supervised learning and BIM.” J. Comput. Civ. Eng. 37 (2): 04022062. https://doi.org/10.1061/JCCEE5.CPENG-5065.
Yang, X.-S. 2010. “Firefly algorithm, stochastic test functions and design optimization.” Int. J. Bio-Inspired Comput. 2 (2): 78–84. https://doi.org/10.1504/IJBIC.2010.032124.
Yang, X.-S. 2012. “Flower pollination algorithm for global optimization.” In Unconventional computation and natural computation, 240–249. Berlin: Springer Nature. https://doi.org/10.1007/978-3-642-32894-7_27.
Zhang, J., S. R. Qian, and C. Tan. 2023. “Automated bridge crack detection method based on lightweight vision models.” Complex Intell. Syst. 9 (2): 1639–1652. https://doi.org/10.1007/s40747-022-00876-6.
Information & Authors
Information
Published In
Journal of Computing in Civil Engineering
Volume 39 • Issue 1 • January 2025
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Dec 23, 2023
Accepted: Jun 11, 2024
Published online: Sep 27, 2024
Published in print: Jan 1, 2025
Discussion open until: Feb 27, 2025
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.