Technical Papers
Jul 12, 2024

A Method for Surveying Road Pavement Distress Based on Front-View Image Data Using a Lightweight Segmentation Approach

Publication: Journal of Computing in Civil Engineering
Volume 38, Issue 5

Abstract

The utilization of low-cost video data is becoming more prevalent in pavement surveys to meet the increasing demand for timely distress detection and repair. Semantic segmentation algorithms can effectively segment pavement features and distresses simultaneously. Previous studies on pavement distress segmentation have primarily focused on cracks, and most multiobjective segmentation algorithms are not accurate or efficient. This paper presents a new method for pavement segmentation using a lightweight network segmentation model that employs DeepLabV3+ with MobileNetV2 as the backbone and a convolution block attention module to extract effective information in the encoder. The authors constructed a self-created data set called ChongQing University Pavement management (CQUPM), which includes five pavement features and six types of distress. Based on the CQUPM data set and a publicly available data set, RTK, the proposed model demonstrates superior accuracy and complexity compared to DeepLabv3+,  U-Net, and Segformer-b3. Its lightweight nature is particularly noteworthy, with a parameter size of only about 1/10 to 1/4 that of other models based on the same data set. The case analysis highlights the exceptional performance of the proposed model, especially in scenarios where multiple types of pavement distress overlap. Furthermore, the model excels in edge segmentation and shows good generalization performance, indicating strong potential for practical applications.

Practical Applications

Maintenance management organizations at the grassroots level, in certain regions or serving specific projects, often face significant daily workloads. Routine survey work is primarily reliant on manual labor due to the high acquisition and operating costs of detection equipment. The segmentation model, trained on a small data set constructed from front-view images, can complete the survey of 2–3 lanes at a time. This model enables the detection of pavement type, pavement marking, and distress information. The model’s excellent generalization capabilities and the small data set lower the technical threshold of the application. This approach can be applied to other transportation infrastructures to address similar management problems. By using low-cost video recording devices to capture video data and quickly construct small data sets, training, and applications based on semantic segmentation techniques, problems can be identified in a timely manner without relying on human labor. This method has strong potential for replication.

Get full access to this article

View all available purchase options and get full access to this article.

Data Availability Statement

All data, models, or codes that support the findings of this study are available from the corresponding author upon reasonable request.

Acknowledgments

This work was supported by the National Natural Science Foundation of China under Grant 51708065.

References

Chatterjee, A., and Y.-C. Tsai. 2020. “Training and testing of smartphone-based pavement condition estimation models using 3D pavement data.” J. Comput. Civ. Eng. 34 (6): 04020043. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000925.
Chen, J., and Y. He. 2022. “A novel U-shaped encoder–decoder network with attention mechanism for detection and evaluation of road cracks at pixel level.” Comput.-Aided Civ. Infrastruct. Eng. 37 (13): 1721–1736. https://doi.org/10.1111/mice.12826.
Chen, L. C., Y. Zhu, G. Papandreou, F. Schroff, and H. Adam. 2018b. “Encoder-decoder with atrous separable convolution for semantic image segmentation.” In Proc., Computer Vision–ECCV 2018: 15th European Conf., Munich, Germany, September 8–14, 2018, Proc., Part VII 15, 833–851. Berlin: Springer.
Chen, L.-C., G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. 2018a. “DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs.” IEEE Trans. Pattern Anal. Mach. Intell. 40 (4): 834–848. https://doi.org/10.1109/TPAMI.2017.2699184.
Chollet F. 2017. “Xception: Deep learning with depthwise separable convolutions.” Preprint, submitted October 7, 2016. http://arxiv.org/abs/1610.02357.
Chun, P., S. Izumi, and T. Yamane. 2021. “Automatic detection method of cracks from concrete surface imagery using two-step light gradient boosting machine.” Comput.-Aided Civ. Infrastruct. Eng. 36 (1): 61–72. https://doi.org/10.1111/mice.12564.
Dong, J., N. Wang, H. Fang, R. Wu, C. Zheng, D. Ma, and H. Hu. 2022. “Automatic damage segmentation in pavement videos by fusing similar feature extraction siamese network (SFE-SNet) and pavement damage segmentation capsule network (PDS-CapsNet).” Autom. Constr. 143 (Nov): 104537. https://doi.org/10.1016/j.autcon.2022.104537.
Du, Y., S. Zhong, H. Fang, N. Wang, C. Liu, D. Wu, Y. Sun, and M. Xiang. 2023. “Modeling automatic pavement crack object detection and pixel-level segmentation.” Autom. Constr. 150 (Jun): 104840. https://doi.org/10.1016/j.autcon.2023.104840.
Du, Z., J. Yuan, F. Xiao, and C. Hettiarachchi. 2021. “Application of image technology on pavement distress detection: A review.” Measurement 184 (Nov): 109900. https://doi.org/10.1016/j.measurement.2021.109900.
El Hakea, A. H., and M. W. Fakhr. 2023. “Recent computer vision applications for pavement distress and condition assessment.” Autom. Constr. 146 (Feb): 104664. https://doi.org/10.1016/j.autcon.2022.104664.
Gong, H., L. Liu, H. Liang, Y. Zhou, and L. Cong. 2024. “A state-of-the-art survey of deep learning models for automated pavement crack segmentation.” Int. J. Transp. Sci. Technol. 13 (Mar): 44–57. https://doi.org/10.1016/j.ijtst.2023.11.005.
Guan, J., X. Yang, L. Ding, X. Cheng, V. C. Lee, and C. Jin. 2021. “Automated pixel-level pavement distress detection based on stereo vision and deep learning.” Autom. Constr. 129 (Sep): 103788. https://doi.org/10.1016/j.autcon.2021.103788.
Howard, A. G., M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam. 2017. “MobileNets: Efficient convolutional neural networks for mobile vision applications.” Preprint, submitted April 17, 2017. http://arxiv.org/abs/1704.04861v1.
Inoue, Y., and H. Nagayoshi. 2020. “Crack detection as a weakly-supervised problem: Towards achieving less annotation-intensive crack detectors.” Preprint, submitted November 4, 2017. http://arxiv.org/abs/2011.02208v1.
Kheradmandi, N., and V. Mehranfar. 2022. “A critical review and comparative study on image segmentation-based techniques for pavement crack detection.” Constr. Build. Mater. 321 (Feb): 126162. https://doi.org/10.1016/j.conbuildmat.2021.126162.
Krizhevsky, A., I. Sutskever, and G. E. Hinton. 2017. “ImageNet classification with deep convolutional neural networks.” Commun. ACM 60 (6): 84–90. https://doi.org/10.1145/3065386.
Lau, S. L. H., E. K. P. Chong, X. Yang, and X. Wang. 2020. “Automated pavement crack segmentation using U-net-based convolutional neural network.” IEEE Access 8 (Jun): 114892–114899. https://doi.org/10.1109/ACCESS.2020.3003638.
Li, Y., C. Liu, Q. Gao, D. Wu, F. Li, and Y. Du. 2022. “ConTrack distress dataset: A continuous observation for pavement deterioration spatio-temporal analysis.” IEEE Trans. Intell. Transp. Syst. 23 (12): 25004–25017. https://doi.org/10.1109/TITS.2022.3201968.
Liu, C., and B. Xu. 2022. “A night pavement crack detection method based on image-to-image translation.” Comput.-Aided Civ. Infrastruct. Eng. 37 (13): 1737–1753. https://doi.org/10.1111/mice.12849.
Liu, J., X. Yang, S. Lau, X. Wang, S. Luo, V. C.-S. Lee, and L. Ding. 2020. “Automated pavement crack detection and segmentation based on two-step convolutional neural network.” Comput.-Aided Civ. Infrastruct. Eng. 35 (11): 1291–1305. https://doi.org/10.1111/mice.12622.
Liu, Y., J. Yao, X. Lu, R. Xie, and L. Li. 2019. “DeepCrack: A deep hierarchical feature learning architecture for crack segmentation.” Neurocomputing 338: 139–153.
Liu, Z., S. Pan, Z. Gao, N. Chen, F. Li, L. Wang, and Y. Hou. 2023. “Automatic intelligent recognition of pavement distresses with limited dataset using generative adversarial networks.” Autom. Constr. 146 (Feb): 104674. https://doi.org/10.1016/j.autcon.2022.104674.
Ma, D., H. Fang, N. Wang, C. Zhang, J. Dong, and H. Hu. 2022. “Automatic detection and counting system for pavement cracks based on PCGAN and YOLO-MF.” IEEE Trans. Intell. Transp. Syst. 23 (11): 22166–22178. https://doi.org/10.1109/TITS.2022.3161960.
Nagaraj, D., M. Mutz, N. George, P. Bansal, and D. Werth. 2022. “A semantic segmentation approach for road defect detection and quantification.” In Proc., ICMVA 2022: 2022 the 5th Int. Conf. on Machine Vision and Applications Singapore, February 18–20, 2022. New York: Association for Computing Machinery. https://doi.org/10.1145/3523111.3523113.
Ning, Z., H. Wang, S. Li, and Z. Xu. 2024. “YOLOv7-RDD: A lightweight efficient pavement distress detection model.” IEEE Trans. Intell. Transp. Syst. (Jan): 1–10. https://doi.org/10.1109/TITS.2023.3347034.
Ranyal, E., A. Sadhu, and K. Jain. 2022. “Road condition monitoring using smart sensing and artificial intelligence: A review.” Sensors 22 (8): 3044. https://doi.org/10.3390/s22083044.
Rateke, T., K. A. Justen, and A. von Wangenheim. 2019. “Road surface classification with images captured from low-cost camera—Road traversing knowledge (RTK) Dataset.” RITA 26 (3): 50–64. https://doi.org/10.22456/2175-2745.91522.
Rateke, T., and A. von Wangenheim. 2021. “Road surface detection and differentiation considering surface damages.” Autom. Rob. 45 (2): 299–312. https://doi.org/10.1007/s10514-020-09964-3.
Ronneberger, O., P. Fischer, and T. Brox. 2015. “U-Net: Convolutional networks for biomedical image segmentation.” Preprint, submitted May 18, 2015. http://arxiv.org/abs/1505.04597v1.
Sandler, M., A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen. 2018. “MobileNetV2: Inverted residuals and linear bottlenecks.” In Proc., IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 4510–4520. New York: IEEE. https://doi.org/10.1109/CVPR.2018.00474.
Sarmiento, J.-A. 2021. “Pavement distress detection and segmentation using YOLOv4 and DeepLabv3 on pavements in the Philippines.” Preprint, submitted March 11, 2021. http://arxiv.org/abs/2103.06467.
Shi, Y., L. Cui, Z. Qi, F. Meng, and Z. Chen. 2016. “Automatic road crack detection using random structured forests.” IEEE Trans. Intell. Transp. Syst. 17 (12): 3434–3445. https://doi.org/10.1109/TITS.2016.2552248.
Sholevar, N., A. Golroo, and S. R. Esfahani. 2022. “Machine learning techniques for pavement condition evaluation.” Autom. Constr. 136 (Apr): 104190. https://doi.org/10.1016/j.autcon.2022.104190.
Simonyan, K., and A. Zisserman. 2014. “Very deep convolutional networks for large-scale image recognition.” Preprint, submitted September 4, 2014. http://arxiv.org/abs/1409.1556.
Szegedy, C., W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. 2014. “Going deeper with convolutions.” Preprint, submitted September 17, 2014. http://arxiv.org/abs/1409.4842.
Tong, Z., T. Ma, J. Huyan, and W. Zhang. 2022. “Pavementscapes: A large-scale hierarchical image dataset for asphalt pavement damage segmentation.” Preprint, submitted July 24, 2022. http://arxiv.org/abs/2208.00775v1.
Wen, T., S. Ding, H. Lang, J. J. Lu, Y. Yuan, Y. Peng, J. Chen, and A. Wang. 2023. “Automated pavement distress segmentation on asphalt surfaces using a deep learning network.” Int. J. Pavement Eng. 24 (2): 2027414. https://doi.org/10.1080/10298436.2022.2027414.
Woo, S., J. Park, J.-Y. Lee, and S. I. Kweon. 2018. “CBAM: Convolutional block attention module.” Preprint, submitted July 17, 2018. http://arxiv.org/abs/1807.06521v2.
Xie, E., W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo. 2021. SegFormer: Simple and efficient design for semantic segmentation with transformers.” Preprint, submitted May 31, 2021. http://arxiv.org/abs/2105.15203v3.
Xu, Y., and Z. Zhang. 2022. “Review of applications of artificial intelligence algorithms in pavement management.” J. Transp. Eng., Part B: Pavements 148 (3): 03122001. https://doi.org/10.1061/JPEODX.0000373.
Zhang, A. A., et al. 2022a. “Intelligent pixel-level detection of multiple distresses and surface design features on asphalt pavements.” Comput.-Aided Civ. Infrastruct. Eng. 37 (13): 1654–1673. https://doi.org/10.1111/mice.12909.
Zhang, C., E. Nateghinia, L. F. Miranda-Moreno, and L. Sun. 2022b. “Pavement distress detection using convolutional neural network (CNN): A case study in Montreal, Canada.” Int. J. Transp. Sci. Technol. 11 (2): 298–309. https://doi.org/10.1016/j.ijtst.2021.04.008.
Zhang, D., Q. Zou, H. Lin, X. Xu, L. He, R. Gui, and Q. Li. 2018a. “Automatic pavement defect detection using 3D laser profiling technology.” Autom. Constr. 96 (Dec): 350–365. https://doi.org/10.1016/j.autcon.2018.09.019.
Zhang, K., H. D. Cheng, and B. Zhang. 2018b. “Unified approach to pavement crack and sealed crack detection using preclassification based on transfer learning.” J. Comput. Civ. Eng. 32 (2): 04018001. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000736.
Zhang, K., Z. Zhang, Y. Luo, and S. Huang. 2017. “Accurate detection and evaluation method for aggregate distribution uniformity of asphalt pavement.” Constr. Build. Mater. 152 (Oct): 715–730. https://doi.org/10.1016/j.conbuildmat.2017.07.058.
Zhong, J., M. Zhang, Y. Ma, R. Xiao, G. Cheng, and B. Huang. 2024. “A multitask fusion network for region-level and pixel-level pavement distress detection.” J. Transp. Eng., Part B: Pavements 150 (1): 04024002. https://doi.org/10.1061/JPEODX.PVENG-1433.
Zou, Q., Y. Cao, Q. Li, Q. Mao, and S. Wang. 2012. “CrackTree: Automatic crack detection from pavement images.” Pattern Recognit. Lett. 33 (3): 227–238. https://doi.org/10.1016/j.patrec.2011.11.004.

Information & Authors

Information

Published In

Go to Journal of Computing in Civil Engineering
Journal of Computing in Civil Engineering
Volume 38Issue 5September 2024

History

Received: Dec 5, 2023
Accepted: Apr 9, 2024
Published online: Jul 12, 2024
Published in print: Sep 1, 2024
Discussion open until: Dec 12, 2024

Permissions

Request permissions for this article.

ASCE Technical Topics:

Authors

Affiliations

Yuanji Yang [email protected]
Master’s Student, College of Artificial Intelligence, Southwest Univ., Chongqing 400715, PR China. Email: [email protected]
Associate Professor, School of Civil Engineering, Chongqing Univ., Chongqing 400045, PR China; Professor, Key Laboratory of New Technology for Construction of Cities in Mountain Area, Ministry of Education, Chongqing Univ., Shabei St. 83, Chongqing 400045, PR China (corresponding author). ORCID: https://orcid.org/0000-0001-7803-7493. Email: [email protected]; [email protected]
Junyang Kang [email protected]
Master’s Student, School of Civil Engineering, Chongqing Univ., Chongqing 400045, PR China. Email: [email protected]
Zhoucong Xu [email protected]
Senior Engineer, China Merchants Chongqing Communications Technology Research & Design Institute Co., Ltd., Xuefu Rd. #33, Nan’an District, Chongqing 400067, PR China. Email: [email protected]

Metrics & Citations

Metrics

Citations

Download citation

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

View Options

Get Access

Access content

Please select your options to get access

Log in/Register Log in via your institution (Shibboleth)
ASCE Members: Please log in to see member pricing

Purchase

Save for later Information on ASCE Library Cards
ASCE Library Cards let you download journal articles, proceedings papers, and available book chapters across the entire ASCE Library platform. ASCE Library Cards remain active for 24 months or until all downloads are used. Note: This content will be debited as one download at time of checkout.

Terms of Use: ASCE Library Cards are for individual, personal use only. Reselling, republishing, or forwarding the materials to libraries or reading rooms is prohibited.
ASCE Library Card (5 downloads)
$105.00
Add to cart
ASCE Library Card (20 downloads)
$280.00
Add to cart
Buy Single Article
$35.00
Add to cart

Get Access

Access content

Please select your options to get access

Log in/Register Log in via your institution (Shibboleth)
ASCE Members: Please log in to see member pricing

Purchase

Save for later Information on ASCE Library Cards
ASCE Library Cards let you download journal articles, proceedings papers, and available book chapters across the entire ASCE Library platform. ASCE Library Cards remain active for 24 months or until all downloads are used. Note: This content will be debited as one download at time of checkout.

Terms of Use: ASCE Library Cards are for individual, personal use only. Reselling, republishing, or forwarding the materials to libraries or reading rooms is prohibited.
ASCE Library Card (5 downloads)
$105.00
Add to cart
ASCE Library Card (20 downloads)
$280.00
Add to cart
Buy Single Article
$35.00
Add to cart

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share