Automated Segmentation and Deterioration Determination of Road Markings
Publication: Journal of Transportation Engineering, Part B: Pavements
Volume 149, Issue 3
Abstract
Road markings are used on pavement surfaces to provide guidance and information for drivers and pedestrians. When navigating road networks, awareness of upcoming hazards and critical information is essential for safe and comfortable driving. However, once the road markings get damaged from traffic application and environmental conditions, it becomes less efficient due to the removal of surface markings. Hence, it is crucial to determine road marking deterioration and propose appropriate rehabilitation methods. With the advancement of deep learning (DL) methods, as a branch of machine learning (ML), there is high potential for developing an automatic road marking detection algorithm with high accuracy and significantly shorter computational and analysis time. In this study, an automated algorithm for segmenting the road markings using mask region-based convolutional neural networks (mask R-CNNs) and determining their deterioration by image processing using the Otsu algorithm was developed. The developed mask R-CNN model used 6,500 and 3,500 images for training and validation, respectively. As a result, the mask R-CNN model could detect and segment road markings with an average accuracy of 97.10% for road marking object detection and 91.0% for road marking segmentation. Furthermore, the image processing algorithm obtained high precision of 92.0%. Therefore, the proposed method was found to be a promising approach to detecting and segmenting road markings together with determining their severity.
Practical Applications
Road markings play a vital role in traffic safety, so their evaluation and immediate rehabilitation are necessary. Recently, studies determining road marking defects have been conducted; however, they are time-consuming and require a retroreflectivity device. These methods, moreover, can only be applied for image classification and not damage evaluation, especially in extreme conditions. This research provides an automated process of determining the deterioration of road markings with high accuracy, addressing the previous issues discussed. In this study, segmenting road markings as new areas using mask R-CNN under extreme environmental and severely damaged conditions is presented. Using the segmented road marking, the damaged area whose pixel value is less than the threshold is calculated and compared with the whole area. The Otsu algorithm then was used to automatically collect the threshold value through the road marking image histogram. The proposed method can be used in different road networks, as applied to a 502-km road in Seoul. This study has great potential to replace manual visual assessments conducted by humans and retroreflectivity devices with higher precision and reliability. Furthermore, this study can aid pavement agencies in easily setting the deterioration criteria of road markings suitable for self-conditions.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, or codes that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This work was supported by the Korea Agency for Infrastructure Technology Advancement (KAIA) grant funded by the Ministry of Land, Infrastructure and Transport (Grant 22POQW-C152342-04) and Sejong University.
References
Alzraiee, H., A. Leal Ruiz, and R. Sprotte. 2021. “Detecting of pavement marking defects using faster R-CNN.” J. Perform. Constr. Facil. 35 (4): 04021035. https://doi.org/10.1061/(ASCE)CF.1943-5509.0001606.
ASTM. 1970. Method of test for night visibility of traffic paints. ASTM D1011-52 (1970). West Conshohocken, PA: ASTM.
Burghardt, T. E., A. Pashkevich, D. Babić, H. Mosböck, D. Babić, and L. Żakowska. 2022. “Microplastics and road markings: The role of glass beads and loss estimation.” Transp. Res. Part D Transp. Environ. 102 (Jan): 103123. https://doi.org/10.1016/j.trd.2021.103123.
Chen, C., S. Chandra, Y. Han, and H. Seo. 2021. “Deep learning-based thermal image analysis for pavement defect detection and classification considering complex pavement conditions.” Remote Sens. 14 (1): 106. https://doi.org/10.3390/rs14010106.
Chen, P.-R., S.-Y. Lo, H.-M. Hang, S.-W. Chan, and J.-J. Lin. 2018. “Efficient road lane marking detection with deep learning.” In Proc., 2018 IEEE 23rd Int. Conf. on Digital Signal Processing (DSP), 1–5. New York: IEEE.
Dutta, A., and A. Zisserman. 2019. “The VIA annotation software for images, audio and video.” Preprint, submitted April 24, 2019. https://arxiv.org/abs/1904.10699.
Ghafoorian, M., C. Nugteren, N. Baka, O. Booij, and M. Hofmann. 2018. “EL-GAN: Embedding loss driven generative adversarial networks for lane detection.” In Vol. 11129 of Proc., European Conf. on Computer Vision (ECCV) Workshops. Berlin: Springer. https://doi.org/10.1007/978-3-030-11009-3_15.
Guo, J., M.-J. Tsai, and J.-Y. Han. 2015. “Automatic reconstruction of road surface features by using terrestrial mobile lidar.” Autom. Constr. 58 (Oct): 165–175. https://doi.org/10.1016/j.autcon.2015.07.017.
He, K., G. Gkioxari, P. Dollár, and R. Girshick. 2017. “Mask R-CNN.” In Proc., IEEE Int. Conf. on Computer Vision, 2961–2969. New York: IEEE.
Highways England. 2020. Inspection and assessment of road markings and road studs. CS 126-2020. Guildford, UK: Highways England.
Hou, Y., Z. Ma, C. Liu, and C. C. Loy. 2019. “Learning lightweight lane detection CNNs by self attention distillation.” In Proc., IEEE/CVF Int. Conf. on Computer Vision, 1013–1021. New York: IEEE.
Huang, Y., S. Chen, Y. Chen, Z. Jian, and N. Zheng. 2018. “Spatial-temporal based lane detection using deep learning.” In Proc., IFIP Int. Conf. on Artificial Intelligence Applications and Innovations, 143–154. Berlin: Springer. https://doi.org/10.1007/978-3-319-92007-8_13.
Killick, R., P. Fearnhead, and I. A. Eckley. 2012. “Optimal detection of changepoints with a linear computational cost.” J. Am. Stat. Assoc. 107 (500): 1590–1598. https://doi.org/10.1080/01621459.2012.737745.
Kopf, J. 2004. Reflectivity of pavement markings: Analysis of retroreflectivity degradation curves. Olympia, WA: Washington State DOT.
KoROAD (Korea Road Traffic Authority). 2012. “Traffic signs.” Road Traffic Authority Driver’s License Examination Office. Accessed June 29, 2021. http://dl.koroad.or.kr/license/en/sub/trafficSigns.html.
Lee, S., J. Kim, J. Shin Yoon, S. Shin, O. Bailo, N. Kim, T.-H. Lee, H. Seok Hong, S.-H. Han, and I. So Kweon. 2017. “VPGNet: Vanishing point guided network for lane and road marking detection and recognition.” In Proc., IEEE Int. Conf. on Computer Vision, 1947–1955. New York: IEEE.
Li, L., W. Luo, and K. C. Wang. 2018. “Lane marking detection and reconstruction with line-scan imaging data.” Sensors 18 (5): 1635. https://doi.org/10.3390/s18051635.
Li, W., Q. Feng, L. Jialun, S. Fengdong, and Y. Wang. 2020. “A lane detection network based on IBN and attention.” Multimedia Tools Appl. 79 (23–24): 16473–16486. https://doi.org/10.1007/s11042-019-7475-x.
McGee, H. W., and D. Mace. 1987. Retroreflectivity of roadway signs for adequate visibility: A guide. Wasington, DC: Transportation Research Board.
Nguyen, S. D., T. S. Tran, V. P. Tran, H. J. Lee, M. J. Piran, and V. P. Le. 2022. “Deep learning-based crack detection: A survey.” Int. J. Pavement Res. Technol. 1–25. https://doi.org/10.1007/s42947-022-00172-z.
Otsu, N. 1979. “A threshold selection method from gray-level histograms.” IEEE Trans. Syst. Man Cybern. 9 (1): 62–66. https://doi.org/10.1109/TSMC.1979.4310076.
Redmon, J., and A. Farhadi. 2018. “YOLOv3: An incremental improvement.” Preprint, submitted April 8, 2018. https://arxiv.org/abs/1804.02767.
Ren, S., K. He, R. Girshick, and J. Sun. 2016. “Faster R-CNN: Towards real-time object detection with region proposal networks.” IEEE Trans. Pattern Anal. Mach. Intell. 39 (6): 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031.
Ruiza, A. L., and H. Alzraieeb. 2020. “Automated pavement marking defects detection.” In Vol. 37 of Proc., Int. Symp. on Automation and Robotics in Construction (ISARC), 67–73. Kitakyushu, Japan: IAARC Publications. https://doi.org/10.22260/ISARC2020/0011.
Sergeev, A., and M. Del Balso. 2018. “Horovod: Fast and easy distributed deep learning in TensorFlow.” Preprint, submitted February 15, 2018. https://arxiv.org/abs/1802.05799.
Sun, Z. 2020. “Vision based lane detection for self-driving car.” In Proc., 2020 IEEE Int. Conf. on Advances in Electrical Engineering and Computer Applications (AEECA), 635–638. New York: IEEE.
Tang, J., S. Li, and P. Liu. 2021. “A review of lane detection methods based on deep learning.” Pattern Recognit. 111 (Mar): 107623. https://doi.org/10.1016/j.patcog.2020.107623.
Tian, Y., J. Gelernter, X. Wang, W. Chen, J. Gao, Y. Zhang, and X. Li. 2018. “Lane marking detection via deep convolutional neural network.” Neurocomputing 280 (Mar): 46–55. https://doi.org/10.1016/j.neucom.2017.09.098.
Tran, T. S., V. P. Tran, H. J. Lee, J. M. Flores, and V. P. Le. 2022. “A two-step sequential automated crack detection and severity classification process for asphalt pavements.” Int. J. Pavement Eng. 23 (6): 2019–2033. https://doi.org/10.1080/10298436.2020.1836561.
Tran, V. P., T. S. Tran, H. J. Lee, K. D. Kim, J. Baek, and T. T. Nguyen. 2021. “One stage detector (RetinaNet)-based crack detection for asphalt pavements considering pavement distresses and surface objects.” J. Civ. Struct. Health Monit. 11 (1): 205–222. https://doi.org/10.1007/s13349-020-00447-8.
Xu, S., J. Wang, P. Wu, W. Shou, X. Wang, and M. Chen. 2021. “Vision-based pavement marking detection and condition assessment—A case study.” Appl. Sci. 11 (7): 3152. https://doi.org/10.3390/app11073152.
Yu, H., Y. Yuan, Y. Guo, and Y. Zhao. 2016. “Vision-based lane marking detection and moving vehicle detection.” In Vol. 2 of Proc., 2016 8th Int. Conf. on Intelligent Human-Machine Systems and Cybernetics (IHMSC), 574–577. New York: IEEE.
Zhang, A., K. C. P. Wang, E. Yang, J. Q. Li, C. Chen, and Y. Qiu. 2018a. “Pavement lane marking detection using matched filter.” Measurement 130 (Dec): 105–117. https://doi.org/10.1016/j.measurement.2018.07.089.
Zhang, J., Y. Xu, B. Ni, and Z. Duan. 2018b. “Geometric constrained joint lane segmentation and lane boundary detection.” In Proc., European Conf. on Computer Vision (ECCV), 486–502. Berlin: Springer. https://doi.org/10.1007/978-3-030-01246-5_30.
Zhang, X., W. Yang, X. Tang, and J. Liu. 2018c. “A fast learning method for accurate and robust lane detection using two-stage feature extraction with YOLO v3.” Sensors 18 (12): 4308. https://doi.org/10.3390/s18124308.
Zou, Q., H. Jiang, Q. Dai, Y. Yue, L. Chen, and Q. Wang. 2019. “Robust lane detection from continuous driving scenes using deep neural networks.” IEEE Trans. Veh. Technol. 69 (1): 41–54. https://doi.org/10.1109/TVT.2019.2949603.
Information & Authors
Information
Published In
Journal of Transportation Engineering, Part B: Pavements
Volume 149 • Issue 3 • September 2023
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Feb 28, 2022
Accepted: Mar 15, 2023
Published online: May 19, 2023
Published in print: Sep 1, 2023
Discussion open until: Oct 19, 2023
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Wenxiu Wu, Xiaoyong Zou, Jing Liu, Yonglin Hu, Zhong Lv, Ruijian Zheng, Zhen Liu, Research on the Anti-Fouling Properties of Double-Coated Road Markings, E3S Web of Conferences, 10.1051/e3sconf/202451203037, 512, (03037), (2024).