Automated Identification of Pavement Structural Distress Using State-of-the-Art Object Detection Models and Nondestructive Testing
Publication: Journal of Computing in Civil Engineering
Volume 38, Issue 4
Abstract
The rapid identification of structural damage in pavements remains a challenging issue. This paper proposes an automatic identification method that combines nondestructive testing, the ground penetrating radar (GPR), and object detection (OD) models for pavement structural assessment. Radar wave image data were collected from a 40 km single-lane track road and processed to create the training data set of 2,000 labeled images with two types of structural distress. Four OD models, including Faster R-CNN, YOLOv5, YOLOv8, and DETR, were adapted and trained on the data set to recognize structural damage in the images, and their prediction performances were compared. The results demonstrated that YOLOv8 was the state-of-the-art (SOTA) model, which achieved precise identification of the location and type of pavement structural damage with a mean average precision (mAP) of 98.5%. This study contributes to the improvement of latest OD models in the non-destructive inspection, facilitating the development of real-time automated pavement assessment and maintenance decision systems.
Practical Applications
This research addresses a common problem in civil engineering: how to quickly detect all hidden damage in pavement structures over long stretches of highway projects. Although there has been previous research in this area, finding a more reliable solution has proven to be a challenging goal. This study presents a method that utilizes the geological radar, a nondestructive testing technology, meaning it will not harm the road, along with artificial intelligence (AI) computer models to identify damage under the pavement surface. Think of it as teaching a computer to recognize cracks or looseness in the radar images of the road. Four different computer models were trained using 2,000 road images with known damage to teach them how to identify different damage features. The results were impressive—the computer could accurately identify the location and type of damage with a high level of confidence. In fact, the best model achieved a 98.5% score measured by mean average precision, which indicates its effectiveness. This research has significant practical applications as it streamlines the process of inspecting roads for structural problems, making it faster and more efficient. It can help with real-time evaluations and decisions about road maintenance.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, or code that support the findings of this study are available from the corresponding author upon reasonable requests.
References
Alsharqawi, M., T. Dawood, S. Abdelkhalek, M. Abouhamad, and T. Zayed. 2022. “Condition assessment of concrete-made structures using ground penetrating radar.” Autom. Constr. 144 (Dec): 104627. https://doi.org/10.1016/j.autcon.2022.104627.
Benedetto, A., and S. Pensa. 2007. “Indirect diagnosis of pavement structural damages using surface GPR reflection techniques.” J. Appl. Geophys. 62 (2): 107–123. https://doi.org/10.1016/j.jappgeo.2006.09.001.
Cano-Ortiz, S., P. Pascual-Muñoz, and D. Castro-Fresno. 2022. “Machine learning algorithms for monitoring pavement performance.” Autom. Constr. 139 (Jul): 104309. https://doi.org/10.1016/j.autcon.2022.104309.
Chen, L., and H. Luo. 2014. “A BIM-based construction quality management model and its applications.” Autom. Constr. 46 (Mar): 64–73. https://doi.org/10.1016/j.autcon.2014.05.009.
Data Mining. 2011. Data mining: Practical machine learning tools and techniques. Amsterdam, Netherlands: Elsevier.
Dong, J., N. Wang, H. Fang, R. Wu, C. Zheng, D. Ma, and H. Hu. 2022. “Automatic damage segmentation in pavement videos by fusing similar feature extraction Siamese network (SFE-SNet) and pavement damage segmentation capsule network (PDS-CapsNet).” Autom. Constr. 143 (Mar): 104537. https://doi.org/10.1016/j.autcon.2022.104537.
Gao, J., D. Yuan, Z. Tong, J. Yang, and D. Yu. 2020. “Autonomous pavement distress detection using ground penetrating radar and region-based deep learning.” Measurement 164 (Mar): 108077. https://doi.org/10.1016/j.measurement.2020.108077.
Garcia, J., G. Villavicencio, F. Altimiras, B. Crawford, R. Soto, V. Minatogawa, M. Franco, D. Martínez-Muñoz, and V. Yepes. 2022. “Machine learning techniques applied to construction: A hybrid bibliometric analysis of advances and future directions.” Autom. Constr. 142 (Mar): 104532. https://doi.org/10.1016/j.autcon.2022.104532.
Girshick, R. 2015. “Fast R-CNN.” In Proc., IEEE Int. Conf. on Computer Vision, 1440–1448. New York: IEEE.
Gong, H., Y. Sun, and B. Huang. 2019. “Gradient boosted models for enhancing fatigue cracking prediction in mechanistic-empirical pavement design guide.” J. Transp. Eng. Part B Pavements 145 (2): 04019014. https://doi.org/10.1061/JPEODX.0000121.
Gong, H., Y. Sun, Z. Mei, and B. Huang. 2018. “Improving accuracy of rutting prediction for mechanistic-empirical pavement design guide with deep neural networks.” Constr. Build. Mater. 190 (May): 710–718. https://doi.org/10.1016/j.conbuildmat.2018.09.087.
Guo, F., Y. Qian, J. Liu, and H. Yu. 2023. “Pavement crack detection based on transformer network.” Autom. Constr. 145 (Feb): 104646. https://doi.org/10.1016/j.autcon.2022.104646.
Guo, X., and P. Hao. 2021a. “Influential factors and evaluation methods of the performance of grouted semi-flexible pavement (GSP)–A review.” Appl. Sci. 11 (15): 6700. https://doi.org/10.3390/app11156700.
Guo, X., and P. Hao. 2021b. “Using a random forest model to predict the location of potential damage on asphalt pavement.” Appl. Sci. 11 (21): 10396. https://doi.org/10.3390/app112110396.
Hamzah, M. O., M. R. Kakar, and M. R. Hainin. 2015. “An overview of moisture damage in asphalt mixtures.” J. Teknol. 73 (4): 125–131. https://doi.org/10.11113/jt.v73.4305.
Huber, E., and G. Hans. 2018. “RGPR–An open-source package to process and visualize GPR data.” In Proc., 2018 17th Int. Conf. on Ground Penetrating Radar (GPR), 1–4. New York: IEEE.
Huyan, J., W. Li, S. Tighe, J. Zhai, Z. Xu, and Y. Chen. 2019. “Detection of sealed and unsealed cracks with complex backgrounds using deep convolutional neural network.” Autom. Constr. 107 (May): 102946. https://doi.org/10.1016/j.autcon.2019.102946.
Jia, X., M. Woods, H. Gong, W. Hu, B. Huang, and B. Faraj. 2019. “Utilization of state performance indices to correlate national performance measures for asphalt pavements in Tennessee.” Transp. Res. Rec. 2673 (6): 379–388. https://doi.org/10.1177/0361198119837500.
Jiang, X., J. Gabrielson, B. Huang, Y. Bai, P. Polaczyk, M. Zhang, W. Hu, and R. Xiao. 2022. “Evaluation of inverted pavement by structural condition indicators from falling weight deflectometer.” Constr. Build. Mater. 319 (Mar): 125991. https://doi.org/10.1016/j.conbuildmat.2021.125991.
Jing, C., J. Zhang, and B. Song. 2020. “An innovative evaluation method for performance of in-service asphalt pavement with semi-rigid base.” Constr. Build. Mater. 235 (Feb): 117376. https://doi.org/10.1016/j.conbuildmat.2019.117376.
Jocher, G., et al. 2020. ultralytics/yolov5: v3.0. Honolulu, HI: Zenodo. https://doi.org/10.5281/zenodo.3983579.
Justo-Silva, R., A. Ferreira, and G. Flintsch. 2021. “Review on machine learning techniques for developing pavement performance prediction models.” Sustainability 13 (9): 5248. https://doi.org/10.3390/su13095248.
Karballaeezadeh, N., H. Ghasemzadeh Tehrani, D. Mohammadzadeh Shadmehri, and S. Shamshirband. 2020. “Estimation of flexible pavement structural capacity using machine learning techniques.” Front. Struct. Civ. Eng. 14 (5): 1083–1096. https://doi.org/10.1007/s11709-020-0654-z.
Khamzin, A. K., A. V. Varnavina, E. V. Torgashov, N. L. Anderson, and L. H. Sneed. 2017. “Utilization of air-launched ground penetrating radar (GPR) for pavement condition assessment.” Constr. Build. Mater. 141 (Feb): 130–139. https://doi.org/10.1016/j.conbuildmat.2017.02.105.
Le Bastard, C., V. Baltazart, X. Dérobert, and Y. Wang. 2012. “Support vector regression method applied to thin pavement thickness estimation by GPR.” In Proc., 2012 14th Int. Conf. on Ground Penetrating Radar (GPR), 349–353. New York: IEEE.
Li, J., T. Liu, X. Wang, and J. Yu. 2022a. “Automated asphalt pavement damage rate detection based on optimized GA-CNN.” Autom. Constr. 136 (Jun): 104180. https://doi.org/10.1016/j.autcon.2022.104180.
Li, J., G. Yin, X. Wang, and W. Yan. 2022b. “Automated decision making in highway pavement preventive maintenance based on deep learning.” Autom. Constr. 135 (Feb): 104111. https://doi.org/10.1016/j.autcon.2021.104111.
Li, Y., P. Hao, C. Zhao, J. Ling, T. Wu, D. Li, J. Liu, and B. Sun. 2021. “Anti-rutting performance evaluation of modified asphalt binders: A review.” J. Traffic Transp. Eng. 8 (3): 339–355.
Li, Y., C. Liu, G. Yue, Q. Gao, and Y. Du. 2022c. “Deep learning-based pavement subsurface distress detection via ground penetrating radar data.” Autom. Constr. 142 (Mar): 104516. https://doi.org/10.1016/j.autcon.2022.104516.
Liu, F., J. Liu, and L. Wang. 2022a. “Deep learning and infrared thermography for asphalt pavement crack severity classification.” Autom. Constr. 140 (Mar): 104383. https://doi.org/10.1016/j.autcon.2022.104383.
Liu, X., P. Hao, A. Wang, L. Zhang, B. Gu, and X. Lu. 2021. “Non-destructive detection of highway hidden layer defects using a ground-penetrating radar and adaptive particle swarm support vector machine.” Peer J. Comput. Sci. 7 (Mar): e417. https://doi.org/10.7717/peerj-cs.417. eCollection 2021.
Liu, Z., X. Gu, W. Wu, X. Zou, Q. Dong, and L. Wang. 2022b. “GPR-based detection of internal cracks in asphalt pavement: A combination method of DeepAugment data and object detection.” Measurement 197 (Jun): 111281. https://doi.org/10.1016/j.measurement.2022.111281.
Mahesh, B. 2018. “Machine learning algorithms—A review.” Int. J. Sci. Res. 9 (1): 7.
Makantasis, K., K. Karantzalos, A. Doulamis, and N. Doulamis. 2015. “Deep supervised learning for hyperspectral data classification through convolutional neural networks.” In Proc., 2015 IEEE Int. Geoscience and Remote Sensing Symp. (IGARSS), 4959–4962. New York: IEEE.
Marecos, V., S. Fontul, M. de Lurdes Antunes, and M. Solla. 2017. “Evaluation of a highway pavement using non-destructive tests: Falling weight deflectometer and ground penetrating radar.” Constr. Build. Mater. 154 (Mar): 1164–1172. https://doi.org/10.1016/j.conbuildmat.2017.07.034.
Miller, J. S., W. Y. Bellinger, and United States Federal Highway Administration, Office of Infrastructure Research and Development. 2003. Distress identification manual for the long-term pavement performance program. 4th ed. Washington, DC: Federal Highway Administration.
Mohamed, O. A., M. Ati, and O. F. Najm. 2017. “Predicting compressive strength of sustainable self-consolidating concrete using random forest.” Key Eng. Mater. 744 (Aug): 141–145. https://doi.org/10.4028/www.scientific.net/KEM.744.141.
Mostafa, K., and T. Hegazy. 2021. “Review of image-based analysis and applications in construction.” Autom. Constr. 122 (May): 103516. https://doi.org/10.1016/j.autcon.2020.103516.
Ouma, Y. O., and M. Hahn. 2017. “Pothole detection on asphalt pavements from 2D-colour pothole images using fuzzy c-means clustering and morphological reconstruction.” Autom. Constr. 83 (Mar): 196–211. https://doi.org/10.1016/j.autcon.2017.08.017.
Pan, Y., and L. Zhang. 2021. “Roles of artificial intelligence in construction engineering and management: A critical review and future trends.” Autom. Constr. 122 (Mar): 103517. https://doi.org/10.1016/j.autcon.2020.103517.
Peraka, N. S. P., and K. P. Biligiri. 2020. “Pavement asset management systems and technologies: A review.” Autom. Constr. 119 (Mar): 103336. https://doi.org/10.1016/j.autcon.2020.103336.
Rasol, M., J. C. Pais, V. Pérez-Gracia, M. Solla, F. M. Fernandes, S. Fontul, D. Ayala-Cabrera, F. Schmidt, and H. Assadollahi. 2022. “GPR monitoring for road transport infrastructure: A systematic review and machine learning insights.” Constr. Build. Mater. 324 (Mar): 126686. https://doi.org/10.1016/j.conbuildmat.2022.126686.
Rasol, M. A., V. Pérez-Gracia, F. M. Fernandes, J. C. Pais, S. Santos-Assunçao, C. Santos, and V. Sossa. 2020. “GPR laboratory tests and numerical models to characterize cracks in cement concrete specimens, exemplifying damage in rigid pavement.” Measurement 158 (Jul): 107662. https://doi.org/10.1016/j.measurement.2020.107662.
Ren, Y. 2021. “Optimizing predictive maintenance with machine learning for reliability improvement.” ASCE-ASME J. Risk Uncertainty Eng. Syst. Part B: Mech. Eng. 7 (3): 030801. https://doi.org/10.1115/1.4049525.
Saarenketo, T., and T. Scullion. 2000. “Road evaluation with ground penetrating radar.” J. Appl. Geophys. 43 (2): 119–138. https://doi.org/10.1016/S0926-9851(99)00052-X.
Sánchez-Silva, M., D. M. Frangopol, J. Padgett, and M. Soliman. 2016. “Maintenance and operation of infrastructure systems: Review.” J. Struct. Eng. 142 (9): F4016004. https://doi.org/10.1061/(ASCE)ST.1943-541X.0001543.
Santos, J., J. Bryce, G. Flintsch, and A. Ferreira. 2017. “A comprehensive life cycle costs analysis of in-place recycling and conventional pavement construction and maintenance practices.” Int. J. Pavement Eng. 18 (8): 727–743. https://doi.org/10.1080/10298436.2015.1122190.
Sebaaly, P., N. Tabatabaee, R. Bonaquist, and D. Anderson. 1989. “Evaluating structural damage of flexible pavements using cracking and falling weight deflectometer data.” Transp. Res. Rec. 1227 (May): 115–127.
Shetty, A. K., I. Saha, R. M. Sanghvi, S. A. Save, and Y. J. Patel. 2021. “A review: Object detection models.” In Proc., 2021 6th Int. Conf. for Convergence in Technology (I2CT), 1–8. New York: IEEE.
Sholevar, N., A. Golroo, and S. R. Esfahani. 2022. “Machine learning techniques for pavement condition evaluation.” Autom. Constr. 136 (Mar): 104190. https://doi.org/10.1016/j.autcon.2022.104190.
Solanki, U. 2014. “A review on structural evaluation of flexible pavements using falling weight deflectometer.” STM J. 2 (Jan): 1–10.
Talvik, O., and A. Aavik. 2009. “Use of FWD Deflection Basin Parameters (SCI, BDI, BCI) for Pavement Condition Assessment.” Balt. J. Road Bridge Eng. 4 (4): 196–202. https://doi.org/10.3846/1822-427X.2009.4.196-202.
Tang, F., C. Han, T. Ma, T. Chen, and Y. Jia. 2021. “Quantitative analysis and visual presentation of segregation in asphalt mixture based on image processing and BIM.” Autom. Constr. 121 (Jan): 103461. https://doi.org/10.1016/j.autcon.2020.103461.
Terven, J., and D. Cordova-Esparza. 2023. “A comprehensive review of YOLO: From YOLOv1 and beyond.” Preprint, submitted April 2, 2023. http://arxiv.org/abs2304.00501.
Torres-Machi, C., A. Osorio-Lird, A. Chamorro, C. Videla, S. L. Tighe, and C. Mourgues. 2018. “Impact of environmental assessment and budgetary restrictions in pavement maintenance decisions: Application to an urban network.” Transp. Res. Part D Transp. Environ. 59 (Mar): 192–204. https://doi.org/10.1016/j.trd.2017.12.017.
Tsai, Y.-C., Y. Zhao, B. Pop-Stefanov, and A. Chatterjee. 2021. “Automatically detect and classify asphalt pavement raveling severity using 3D technology and machine learning.” Int. J. Pavement Res. Technol. 14 (4): 487–495. https://doi.org/10.1007/s42947-020-0138-5.
Varela-González, M., M. Solla, J. Martínez-Sánchez, and P. Arias. 2014. “A semi-automatic processing and visualisation tool for ground-penetrating radar pavement thickness data.” Autom. Constr. 45 (Sep): 42–49. https://doi.org/10.1016/j.autcon.2014.05.004.
Wiley, V., and T. Lucas. 2018. “Computer vision and image processing: A paper review.” Int. J. Artif. Intell. Res. 2 (1): 22–36. https://doi.org/10.29099/ijair.v2i1.42.
Xiao, M., R. Luo, and X. Yu. 2022. “Assessment of asphalt pavement overall performance condition using functional indexes and FWD deflection basin parameters.” Constr. Build. Mater. 341 (Jul): 127872. https://doi.org/10.1016/j.conbuildmat.2022.127872.
Yao, H., Y. Liu, X. Li, Z. You, Y. Feng, and W. Lu. 2022. “A detection method for pavement cracks combining object detection and attention mechanism.” IEEE Trans. Intell. Transp. Syst. 23 (11): 22179–22189. https://doi.org/10.1109/TITS.2022.3177210.
Zhang, X.-D. 2020. “Machine learning.” In A matrix algebra approach to artificial intelligence, 223–440. Singapore: Springer Singapore.
Zhao, S., I. L. Al-Qadi, and S. Wang. 2018. “Prediction of thin asphalt concrete overlay thickness and density using nonlinear optimization of GPR data.” NDT & E Int. 100: 20–30. https://doi.org/10.1016/j.ndteint.2018.08.001.
Zhu, J., J. Zhong, T. Ma, X. Huang, W. Zhang, and Y. Zhou. 2022. “Pavement distress detection using convolutional neural networks with images captured via UAV.” Autom. Constr. 133 (Jan): 103991. https://doi.org/10.1016/j.autcon.2021.103991.
Zhu, X., W. Su, L. Lu, B. Li, X. Wang, and J. Dai. 2021. “Deformable DETR: Deformable transformers for end-to-end object detection.” Preprint, submitted October 8, 2020. http://arxiv.org/abs/2010.04159.
Information & Authors
Information
Published In
Journal of Computing in Civil Engineering
Volume 38 • Issue 4 • July 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Dec 3, 2023
Accepted: Jan 10, 2024
Published online: Mar 27, 2024
Published in print: Jul 1, 2024
Discussion open until: Aug 27, 2024
ASCE Technical Topics:
- Automatic identification systems
- Detection methods
- Engineering fundamentals
- Equipment and machinery
- Field tests
- Geotechnical engineering
- Geotechnical investigation
- Gravels
- Infrastructure
- Methodology (by type)
- Models (by type)
- Nondestructive tests
- Pavement condition
- Pavements
- Penetration tests
- Radar
- Structural engineering
- Structural models
- Structural system identification
- Structural systems
- Tests (by type)
- Transportation engineering
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.