LiDAR-Based Automatic Pavement Distress Detection and Management Using Deep Learning and BIM
Publication: Journal of Construction Engineering and Management
Volume 150, Issue 7
Abstract
Due to the progress in light detection and ranging (LiDAR) technology, the collection of road point cloud data containing depth information and spatial coordinates has become more accessible. Consequently, utilizing point cloud data for pavement distress detection and quantification emerges as a crucial approach to improving the precision and reliability of road maintenance procedures. This paper aims to automatically detect and visualize pavement distress using LiDAR, deep learning-based 3D object detection method, and building information modeling (BIM). A pavement distress data set is first established using the point cloud data obtained from LiDAR. Then, the 3D object detection network, namely PointPillar, is employed for pavement distress detection, and the detection results will be quantified at a region-level. Finally, pavement BIM model integrating parametrically modeled distress families is built to visually manage the detected distress. After training and validating the model with the pavement distress data set, a detection performance index of recall is 78.5%, mean average precision (mAP) is 62.7%, which is better than other compared point cloud-based methods though the detection performance can be further improved. In addition, a newly untrained section of road is applied for the experiment. The detected distress is integrated in BIM environment for a visual management, providing a better maintenance guidance.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This research is supported by both the Shenzhen Science and Technology Program (No. JCYJ20220818095608018) and the Foundation for Basic and Applied Basic Research of Guangdong Province (No. 2020A1515111189).
References
Bosurgi, G., C. Celauro, O. Pellegrino, N. Rustica, and S. Giuseppe. 2020. “The BIM (building information modeling)-based approach for road pavement maintenance.” In Proc., 5th Int. Symp. on Asphalt Pavements and Environment (APE). Berlin: Springer.
Bosurgi, G., O. Pellegrino, and G. Sollazzo. 2021. “Pavement condition information modelling in an I-BIM environment.” Int. J. Pavement Eng. 23 (13): 1–16. https://doi.org/10.1080/10298436.2021.1978442.
Changxia, M. A., Z. Chunxia, D. I. Feng, and L. I. Mingxian. 2011. “Road crack detection under natural environment.” J. Eng. Graphics 32 (4): 20–26.
Chen, X., H. Ma, J. Wan, B. Li, and T. Xia. 2017. “Multi-view 3D object detection network for autonomous driving.” In Proc., 2017 IEEE Conf. on Computer Vision and Pattern Recognition. New York: IEEE.
Chong, H. Y., R. Lopez, J. Wang, X. Y. Wang, and Z. Y. Zhao. 2016. “Comparative analysis on the adoption and use of BIM in road infrastructure projects.” J. Manage. Eng. 32 (6): 05016021. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000460.
D’Amico, F., A. Di Benedetto, L. B. Ciampoli, L. Bertolini, and A. Napolitano. 2022. “Integrating non-destructive surveys into a preliminary BIM-oriented digital model for possible future application in road pavements management.” Infrastructures 7 (1): 10. https://doi.org/10.3390/infrastructures7010010.
Deng, T., and Y. Tan. 2023. “Efficient pavement distress detection and visual management in lean construction based on BIM and deep learning.” In Proc., 31st Annual Conf. Int. Group for Lean Construction (IGLC31), 174–185. Lille, France: International Group for Lean Construction. https://doi.org/10.24928/2023/0230.
Du, Y. C., N. Pan, Z. H. Xu, F. W. Deng, Y. Shen, and H. Kang. 2021. “Pavement distress detection and classification based on YOLO network.” Int. J. Pavement Eng. 22 (13): 1659–1672. https://doi.org/10.1080/10298436.2020.1714047.
Fei, Y., K. C. P. Wang, A. Zhang, C. Chen, J. Q. Li, Y. Liu, G. Yang, and B. Li. 2020. “Pixel-level cracking detection on 3D asphalt pavement images through deep-learning- based CrackNet-V.” IEEE Trans. Intell. Transp. Syst. 21 (1): 273–284. https://doi.org/10.1109/TITS.2019.2891167.
Gavilan, M., D. Balcones, O. Marcos, D. F. Llorca, M. A. Sotelo, I. Parra, M. Ocana, P. Aliseda, P. Yarza, and A. Amirola. 2011. “Adaptive road crack detection system by pavement classification.” Sensors 11 (10): 9628–9657. https://doi.org/10.3390/s111009628.
Geiger, A., P. Lenz, and R. Urtasun. 2012. “Are we ready for autonomous driving? The KITTI vision benchmark suite.” In Proc., 2012 IEEE Conf. on Computer Vision and Pattern Recognition. New York: IEEE.
Guan, J., X. Yang, L. Ding, X. Cheng, V. C. S. Lee, and C. Jin. 2021. “Automated pixel-level pavement distress detection based on stereo vision and deep learning.” Autom. Constr. 129 (Sep): 103788. https://doi.org/10.1016/j.autcon.2021.103788.
Hadjidemetriou, G. M., P. A. Vela, and S. E. Christodoulou. 2017. “Automated pavement patch detection and quantification using support vector machines.” J. Comput. Civ. Eng. 32 (1): 04017073. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000724.
Han, C., T. Ma, F. Tang, L. Gu, and Z. Tong. 2022. “Construction quality evaluation of asphalt pavement based on BIM and GIS.” Autom. Constr. 141 (Sep): 104398. https://doi.org/10.1016/j.autcon.2022.104398.
Ibragimov, E., H. J. Lee, J. J. Lee, and N. Kim. 2022. “Automated pavement distress detection using region based convolutional neural networks.” Int. J. Pavement Eng. 23 (6): 1981–1992. https://doi.org/10.1080/10298436.2020.1833204.
Jiang, C. L., and Y. J. Tsai. 2015. “Enhanced crack segmentation algorithm using 3D pavement data.” J. Comput. Civ. Eng. 30 (3): 04015050. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000526.
Lang, A. H., S. Vora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom. 2019. “PointPillars: Fast encoders for object detection from point clouds.” In Proc., 2019 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. New York: IEEE.
Lee, B. J., and H. Lee. 2004. “Position-invariant neural network for digital pavement crack analysis.” Comput.-Aided Civ. Infrastruct. Eng. 19 (2): 105–118. https://doi.org/10.1111/j.1467-8667.2004.00341.x.
Lei, X., C. Liu, L. Li, and G. Wang. 2020. “Automated pavement distress detection and deterioration analysis using street view map.” IEEE Access 8 (Apr): 76163–76172. https://doi.org/10.1109/ACCESS.2020.2989028.
Li, B., K. C. P. Wang, Allen Zhang, Enhui Yang, and Guolong Wang. 2020. “Automatic classification of pavement crack using deep convolutional neural network.” Int. J. Pavement Eng. 21 (4): 457–463. https://doi.org/10.1080/10298436.2018.1485917.
Li, J., T. Liu, and X. Wang. 2022. “Advanced pavement distress recognition and 3D reconstruction by using GA-DenseNet and binocular stereo vision.” Measurement 201 (Sep): 111760. https://doi.org/10.1016/j.measurement.2022.111760.
Liang, X., X. Yu, C. Chen, Y. Jin, and J. Huang. 2022. “Automatic classification of pavement distress using 3D ground-penetrating radar and deep convolutional neural network.” IEEE Trans. Intell. Transp. Syst. 23 (11): 22269–22277. https://doi.org/10.1109/TITS.2022.3197712.
Lin, Y. C. 2014. “Construction 3D BIM-based knowledge management system: A case study.” J. Civ. Eng. Manage. 20 (2): 186–200. https://doi.org/10.3846/13923730.2013.801887.
Liu, W., D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, and A. C. Berg. 2016. “SSD: Single shot multibox detector.” In Proc., Computer Vision-ECCV 2016: 14th European Conf., 21–37. Berlin: Springer.
Medina, R., J. Llamas, E. Zalama, J. Gomez-Garcia-Bermejo, and IEEE. 2014. “Enhanced automatic detection of road surface cracks by combining 2D/3D image processing techniques.” In Proc., 2014 IEEE Int. Conf. on Image Processing. New York: IEEE.
Ouyang, W., and B. Xu. 2013. “Pavement cracking measurements using 3D laser-scan images.” Meas. Sci. Technol. 24 (10): 105204. https://doi.org/10.1088/0957-0233/24/10/105204.
Porwal, A., and K. N. Hewage. 2013. “Building Information Modeling (BIM) partnering framework for public construction projects.” Autom. Constr. 31 (May): 204–214. https://doi.org/10.1016/j.autcon.2012.12.004.
Qi, C. R., H. Su, K. C. Mo, L. J. Guibas, and IEEE. 2017. “PointNet: Deep learning on point sets for 3D classification and segmentation.” In Proc., 30TH IEEE Conf. on Computer Vision and Pattern Recognition (CVPR 2017). New York: IEEE.
Ranyal, E., A. Sadhu, and K. Jain. 2023. “Automated pothole condition assessment in pavement using photogrammetry-assisted convolutional neural network.” Int. J. Pavement Eng. 24 (1): 2183401. https://doi.org/10.1080/10298436.2023.2183401.
Roberts, R., F. Menant, G. Di Mino, and V. Baltazart. 2022. “Optimization and sensitivity analysis of existing deep learning models for pavement surface monitoring using low-quality images.” Autom. Constr. 140 (Aug): 104332. https://doi.org/10.1016/j.autcon.2022.104332.
Shah, T. 2018. “Measuring object detection models—mAP—What is mean average precision.” Tarang Shah-Blog 26: 104332.
Shi, S., C. Guo, L. Jiang, Z. Wang, J. Shi, X. Wang, and H. Li. 2020. “PV-RCNN: Point-Voxel feature set abstraction for 3D object detection.” In Proc., IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR). New York: IEEE.
Shi, S., X. Wang, and H. Li. 2019. “Pointrcnn: 3D object proposal generation and detection from point cloud.” In Proc., IEEE/CVF Conf. on Computer Vision and Pattern Recognition. New York: IEEE.
Tang, F., T. Ma, C. Han, T. Chen, and Y. Jia. 2021. “Quantitative analysis and visual presentation of segregation in asphalt mixture based on image processing and BIM.” Autom. Constr. 121 (Jan): 103461. https://doi.org/10.1016/j.autcon.2020.103461.
Tang, F. L., T. Ma, J. H. Zhang, Y. S. Guan, and L. F. Chen. 2020. “Integrating three-dimensional road design and pavement structure analysis based on BIM.” Autom. Constr. 113 (Jun): 103152. https://doi.org/10.1016/j.autcon.2020.103152.
Wang, J., X. Y. Wang, W. C. Shou, H. Y. Chong, and J. Guo. 2016. “Building information modeling-based integration of MEP layout designs and constructability.” Autom. Constr. 61 (Jan): 134–146. https://doi.org/10.1016/j.autcon.2015.10.003.
Wang, S. C., and W. S. Tang. 2011. “Pavement crack segmentation algorithm based on local optimal threshold of cracks density distribution.” In Proc., 7th Int. Conf. Advanced Intelligent Computing. Berlin: Springer.
Xiao, L., B. Dai, D. X. Liu, D. W. Zhao, and T. Wu. 2016. “Monocular road detection using structured random forest.” Int. J. Adv. Rob. Syst. 13 (3): 101. https://doi.org/10.5772/63561.
Xu, Q., and G. K. Chang. 2013. “Evaluation of intelligent compaction for asphalt materials.” Autom. Constr. 30 (Mar): 104–112. https://doi.org/10.1016/j.autcon.2012.11.015.
Yan, Y., Y. Mao, and B. Li. 2018. “Second: Sparsely embedded convolutional detection.” Sensors 18 (10): 3337.
Zhang, A., K. C. P. Wang, and C. Ai. 2017a. “3D shadow modeling for detection of descended patterns on 3D pavement surface.” J. Comput. Civ. Eng. 31 (4): 04017019. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000661.
Zhang, A., K. C. P. Wang, E. Y. Baoxian Li, Y. Peng, X. Dai, Y. Liu, Y. Fei, J. Q. Li, and C. Chen. 2017b. “Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network.” Comput. Aided Civ. Infrastruct. Eng. 32 (10): 805–819. https://doi.org/10.1111/mice.12297.
Zhang, A., K. C. P. Wang, Y. Fei, Y. Liu, C. Chen, G. W. Yang, J. Q. Li, E. H. Yang, and S. Qiu. 2019. “Automated pixel-level pavement crack detection on 3D asphalt surfaces with a recurrent neural network.” Comput.-Aided Civ. Infrastruct. Eng. 34 (3): 213–229. https://doi.org/10.1111/mice.12409.
Zhang, A., K. C. P. Wang, R. Ji, and Q. Joshua Li. 2016a. “Efficient system of cracking-detection algorithms with 1-mm 3D-surface models and performance measures.” J. Comput. Civ. Eng. 30 (6): 04016020. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000581.
Zhang, L., F. Yang, Y. D. Zhang, Y. J. Zhu, and IEEE. 2016b. “Road crack detection using deep convolutional neural network.” In Proc., 2016 IEEE Int. Conf. on Image Processing. New York: IEEE.
Zhang, Y., C. Chen, Q. Wu, Q. Lu, S. Zhang, G. Zhang, and Y. Yang. 2018. “A kinect-based approach for 3D pavement surface reconstruction and cracking recognition.” IEEE Trans. Intell. Transp. Syst. 19 (12): 3935–3946. https://doi.org/10.1109/TITS.2018.2791476.
Zhong, J., J. Huyan, J. Zhu, T. Ma, and W. Zhang. 2022. “Multi-scale feature fusion network for pixel-level pavement distress detection.” Autom. Constr. 141 (Sep): 104436. https://doi.org/10.1016/j.autcon.2022.104436.
Zhou, Y., and O. Tuzel. 2018. “VoxelNet: End-to-end learning for point cloud based 3D object detection.” In Proc., 2018 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. New York: IEEE.
Zou, Q., Y. Cao, Q. Q. Li, Q. Z. Mao, and S. Wang. 2012. “Crack tree: Automatic crack detection from pavement images.” Pattern Recognit. Lett. 33 (3): 227–238. https://doi.org/10.1016/j.patrec.2011.11.004.
Zou, Q., Z. Zhang, Q. Li, X. Qi, Q. Wang, and S. Wang. 2019. “DeepCrack: Learning hierarchical convolutional features for crack detection.” IEEE Trans. Image Process. 28 (3): 1498–1512. https://doi.org/10.1109/TIP.2018.2878966.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 150 • Issue 7 • July 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Aug 12, 2023
Accepted: Jan 16, 2024
Published online: May 3, 2024
Published in print: Jul 1, 2024
Discussion open until: Oct 3, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.