Detecting of Pavement Marking Defects Using Faster R-CNN
Publication: Journal of Performance of Constructed Facilities
Volume 35, Issue 4
Abstract
Pavement markings on roads and highways are used to guide the roadway users. They play an essential role in promoting efficient use of the roadway and drivers’ safety. Typically, pavement markings deteriorate at a higher rate and last between 0.5 and 3 years. Because of the short lifecycle, pavement markings require frequent inspection and maintenance. Traditionally, pavement markings have been assessed periodically by road inspectors. This manual method is time-consuming, subjective, and exposes the road inspectors to high safety risks. Therefore, this paper presents a deep learning framework for automated pavement marking defects identification. The proposed framework uses a photogrammetry data set collected from Google Maps. Images of pavement markings are processed by annotating the marking defects. A deep learning algorithm called faster region convolutional neural networks (R-CNN) has been utilized to identify the pavement marking defects. The proposed model went through three iterations of training and used 1,040 annotated images. In the final stage, the model was tested using 60 images and was run for 46,194 epochs. The model was able to identify the pavement marking defects with a confidence level ranging from 43% to 99%. The model result was validated visually by inspecting the condition of the road markings used in testing the model. The proposed automated process is capable of generating a summary report of the condition of pavement markings that can enhance the current practices.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or codes that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This research is supported by the Research, Scholarly, and Creative Activities Grant and Summer Undergraduate Research program from California Polytechnic State University.
References
Adu-Gyamfi, Y. O., T. Tienaah, N. O. Attoh-Okine, and C. Kambhamettu. 2014. “Functional evaluation of pavement condition using a complete vision system.” J. Transp. Eng. 140 (9): 04014040. https://doi.org/10.1061/(ASCE)TE.1943-5436.0000638.
Agent, K. R. 1980. Transverse pavement markings for speed control and accident reduction (abridgement). Washington, DC: Transportation Research Board.
Bali, S., A. Potts, A. Fee, I. Taylor, and J. Glennon. 1978. Cost-effectiveness and safety of alternative roadway delineation treatments for rural two-lane highways. Washington, DC: Federal Highway Administration.
Basile, A. J. 1962. “Effect of pavement edge markings on traffic accidents in Kansas.” In Highway research board bulletin 308, 80–86. Washington, DC: National Research Council.
Carlson, P. J., E. S. Park, and C. K. Andersen. 2009. Benefits of pavement markings. Washington, DC: Transportation Research Board.
Carlson, P. J., G. Schertz, C. Satterfield, K. W. Falk, and T. Taylor. 2014. Methods for maintaining pavement marking retroreflectivity. Washington, DC: Federal Highway Administration.
Chen, P. R., S. Y. Lo, H. M. Hang, S. W. Chan, and J. J. Lin. 2019. “Efficient road lane marking detection with deep learning.” In Proc., IEEE 23rd Int. Conf. on Digital Signal Processing, 1–5. New York: IEEE.
Cheng, M., H. Zhang, C. Wang, and J. Li. 2017. “Extraction and classification of road markings using mobile laser scanning point clouds.” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10 (3): 1182–1196. https://doi.org/10.1109/JSTARS.2016.2606507.
Chimba, D., E. Kidando, and M. Onyango. 2018. “Evaluating the service life of thermoplastic pavement markings: Stochastic approach.” J. Transp. Eng. 144 (3): 04018029. https://doi.org/10.1061/JPEODX.0000055.
Cho, Y., K. Kabassi, J.-H. Pyeon, K. Choi, C. Wang, and T. Norton. 2013. “Effectiveness study of methods for removing temporary pavement markings in roadway construction zones.” J. Constr. Eng. Manage. 139 (3): 257–266. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000608.
Cottrell, B. H., and R. A. Hanson. 2001. Determining the effectiveness of pavement marking materials. Charlottesville, VA: Virginia Transportation Research Council.
Ersoz, A. B., O. Pekcan, and T. Teke. 2017. “Unmanned aerial vehicle based pavement crack identification method integrated with geographic information systems.” In Proc., Transportation Research Board 96th Annual Meeting. Washington, DC: Transportation Research Board.
Forconstructionpros. 2016. State DOTs using drones to improve safety, collect data and cut costs. Washington, DC: American Association of State Highway and Transportation Officials.
Hui, X., J. Bian, X. Zhao, and M. Tan. 2018. “Deep-learning-based autonomous navigation approach for UAV transmission line inspection.” In Proc., 2018 10th Int. Conf. on Advanced Computational Intelligence, ICACI 2018, 455–460. New York: IEEE.
Inzerillo, L., G. Di Mino, and R. Roberts. 2018. “Image-based 3D reconstruction using traditional and UAV datasets for analysis of road pavement distress.” Autom. Constr. 96 (Dec): 457–469. https://doi.org/10.1016/j.autcon.2018.10.010.
Jalayer, M., M. O’Connell, H. Zhou, P. Szary, and S. Das. 2019. “Application of unmanned aerial vehicles to inspect and inventory interchange assets to mitigate wrong-way entries.” ITE J. 89 (7): 36–42.
Khan, M. A., W. Ectors, T. Bellemans, D. Janssens, and G. Wets. 2017. “UAV-based traffic analysis: A universal guiding framework based on literature survey.” Transp. Res. Procedia 22 (Jan): 541–550. https://doi.org/10.1016/j.trpro.2017.03.043.
Maeda, H., Y. Sekimoto, T. Seto, T. Kashiyama, and H. Omata. 2018. “Road damage detection and classification using deep neural networks with smartphone images.” Comput.-Aided Civ. Infrastruct. Eng. 33 (12): 1127–1141. https://doi.org/10.1111/mice.12387.
Mandavilli, S., A. T. McCartt, and R. A. Retting. 2009. “Crash patterns and potential engineering countermeasures at Maryland roundabouts.” Traffic Inj. Prev. 10 (1): 44–50. https://doi.org/10.1080/15389580802485938.
Miller, T. 1991. Benefit-cost analysis of lane marking, 38–45. Washington, DC: National Research Council.
Montebello, D., and J. Schroeder. 2000. Cost of pavement marking materials. Washington, DC: Transporation Research Board.
Morena, D., and T. Leix. 2012. Where these drivers went wrong. Washington, DC: US DOT.
Musick, J. V. 1960. “Effect of pavement edge marking on two-lane rural state highways in Ohio.” In Highway research board bulletin 266, 1–7. Washington, DC: Transporation Research Board.
Pike, A. M., and B. Bommanayakanahalli. 2018. “Development of a pavement marking life cycle cost tool.” Transp. Res. Rec. 2672 (12): 148–157. https://doi.org/10.1177/0361198118758012.
Ren, S., K. He, R. Girshick, and J. Sun. 2015. “Faster R-CNN: Towards real-time object detection with region proposal networks.” Preprint, submitted June 4, 2015. https://arxiv.org/abs/1506.01497.
Retting, R. A., and C. M. Farmer. 1998. Use of pavement markings to reduce excessive traffic speeds on hazardous curves. Washington, DC: Institute of Transportation Engineers.
Simonyan, K., and A. Zisserman. 2014. “Very deep convolutional networks for large-scale image recognition.” Preprint, submitted September 4, 2014. https://arxiv.org/abs/1409.1556.
Stacy, A. R. 2019. Evaluation of machine vision collected pavement marking quality data for use in transportation asset management. College Station, TX: Texas A&M Univ.
Upshaw, P., J. Sevearance, and H. Lee. 2014. Operations manual for mobile retroreflectivity units. Gainesville, FL: Florida DOT.
Xu, S., J. Wang, P. Wu, W. Shou, T. Fang, and X. Wang. 2020. “Vision-based pavement marking detection—A case study.” In Proc., 18th Int. Conf. on Computing in Civil and Building Engineering, edited by E. Toledo Santos and S. Scheer, 1162–1171. Berlin: Springer.
Zeggada, A., F. Melgani, and Y. Bazi. 2017. “A deep learning approach to UAV image multilabeling.” IEEE Geosci. Remote Sens. Lett. 14 (5): 694–698. https://doi.org/10.1109/LGRS.2017.2671922.
Zeiler, M. D., and R. Fergus. 2014. Visualizing and understanding convolutional networks, 818–833. Berlin: Springer.
Zhang, A., K. C. P. Wang, Y. Fei, Y. Liu, C. Chen, G. Yang, J. Q. Li, E. Yang, and S. Qiu. 2019. “Automated pixel-level pavement crack detection on 3D asphalt surfaces with a recurrent neural network.” Comput.-Aided Civ. Infrastruct. Eng. 34 (3): 213–229. https://doi.org/10.1111/mice.12409.
Zhang, C. 2008. “An UAV-based photogrammetric mapping system for road condition assessment.” Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 37: 627–632.
Zhang, H., J. Li, M. Cheng, and C. Wang. 2016a. “Rapid inspection of pavement markings using mobile lidar point clouds.” Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 41: 717–723. https://doi.org/10.5194/isprs-archives-XLI-B1-717-2016.
Zhang, L., F. Yang, Y. D. Zhang, and Y. J. Zhu. 2016b. “Road crack detection using deep convolutional neural network.” In Proc., IEEE Int. Conf. on Image Processing, 3708–3712. New York: IEEE.
Zhang, Y., and D. Wu. 2010. “Methodologies to predict service lives of pavement marking materials.” J. Transp. Res. Forum 45 (3). https://doi.org/10.5399/osu/jtrf.45.3.601.
Information & Authors
Information
Published In
Journal of Performance of Constructed Facilities
Volume 35 • Issue 4 • August 2021
Copyright
© 2021 American Society of Civil Engineers.
History
Received: Dec 16, 2020
Accepted: Mar 15, 2021
Published online: Jun 2, 2021
Published in print: Aug 1, 2021
Discussion open until: Nov 2, 2021
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.