Technical Papers
May 16, 2024

Road Surface Condition Monitoring in Extreme Weather Using a Feature-Learning Enhanced Mask–RCNN

Publication: Journal of Transportation Engineering, Part B: Pavements
Volume 150, Issue 3

Abstract

Road surface condition (RSC) is an important indicator in road safety studies, enabling transportation departments to employ it for conducting surveys, inspections, cleaning, and maintenance, ultimately contributing to improved performance in road upkeep. However, traditional recognition methods can be easily affected when extreme weather frequently occurs such as winter seasonal changes. To achieve real-time and automatic RSC monitoring, this paper proposes an improved Mask–region-based convolutional neural network (RCNN) based on Swin Transformer-PAFPN and a dynamic head detection network. Meanwhile, transfer learning is used to reduce training time, and data enhancement and multiscale training are applied to achieve better performance. The experimental results show that the proposed model achieves an outstanding mean average precision at 0.5 ([email protected]) score of 89.8 under favorable weather conditions characterized by clear visibility, surpassing other popular methods. Notably, the proposed model exhibits lower parameters and GigaFLOPS (GFLOPs) (72.41 and 158.35, respectively) compared to other popular methods, thus demanding fewer computational resources. Furthermore, in challenging weather conditions characterized by poor visibility, such as foggy and nighttime scenarios, the proposed model achieves [email protected] scores of 78.50 and 82.40, respectively. These scores not only outperform those of other popular methods but also underscore the robustness of the proposed model in extreme weather conditions. This exceptional performance demonstrates the proposed model’s effectiveness in addressing complex road conditions under various meteorological circumstances, providing reliable technical support for practical traffic monitoring and road maintenance.

Get full access to this article

View all available purchase options and get full access to this article.

Data Availability Statement

All data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.

Acknowledgments

This study was supported by the National Natural Science Foundation of China under grant no. 62103177. The authors would also like to thank the Ministry of Transportation Ontario Canada for technical support in operating the road/weather information system.

References

Aki, M., T. Rojanaarpa, K. Nakano, Y. Suda, N. Takasuka, T. Isogai, and T. Kawai. 2016. “Road surface recognition using laser radar for automatic platooning.” IEEE Trans. Intell. Transp. Syst. 17 (10): 2800–2810. https://doi.org/10.1109/TITS.2016.2528892.
Bello, I., B. Zoph, A. Vaswani, J. Shlens, and Q. V. Le. 2019. “Attention augmented convolutional networks.” In Proc., IEEE/CVF Int. Conf. on Computer Vision. New York: IEEE.
Bi, J., Z. Zhu, and Q. Meng. 2021. “Transformer in computer vision.” In Proc., 2021 IEEE Int. Conf. on Computer Science, Electronic Information Engineering and Intelligent Control Technology (CEI), 178–188. New York: IEEE.
Buslaev, A., V. I. Iglovikov, E. Khvedchenya, A. Parinov, M. Druzhinin, and A. A. Kalinin. 2020. “Albumentations: Fast and flexible image augmentations.” Information 11 (2): 125. https://doi.org/10.3390/info11020125.
Cai, Z., and N. Vasconcelos. 2018. “Cascade r-cnn: Delving into high quality object detection.” In Proc., IEEE Computer Vision and Pattern Recognition Conf., 6154–6162. New York: IEEE.
Can, S. 2014. Population and dwelling counts, for census metropolitan areas, 2011 and 2006 censuses. Ottawa: Statistics Canada.
Casado-García, Á., and J. Heras. 2020. “Ensemble methods for object detection.” In Proc., 24th European Conf. on Artificial Intelligence, 2688–2695. Santiago, Chile: IOS Press.
Chen, Q., G. Pan, L. Zhao, J. Fan, W. Chen, and A. Zhang. 2023. “An adaptive hybrid attention based convolutional neural net for intelligent transportation object recognition.” IEEE Trans. Intell. Transp. Syst. 24 (7): 7791–7801. https://doi.org/10.1109/TITS.2022.3227245.
Dosovitskiy, A., et al. 2020. “An image is worth 16x16 words: Transformers for image recognition at scale.” Preprint, submitted October 22, 2020. http://arxiv.org/abs/2010.11929.
Gui, K., L. Ye, J. Ge, F. A. Cheikh, and L. Huang. 2019. “Road surface condition detection utilizing resonance frequency and optical technologies.” Sens. Actuators, A 297 (Feb): 111540. https://doi.org/10.1016/j.sna.2019.111540.
Han, K., et al. 2022. “A survey on vision transformer.” IEEE Trans. Pattern Anal. Mach. Intell. 45 (1): 87–110. https://doi.org/10.1109/TPAMI.2022.3152247.
Hatamizadeh, A., V. Nath, Y. Tang, D. Yang, H. R. Roth, and D. Xu. 2021. “Swin unetr: Swin transformers for semantic segmentation of brain tumors in MRI images.” In Proc., 24th Int. MICCAI Brainlesion Workshop. New York: Springer.
He, K., G. Gkioxari, P. Dollár, and R. Girshick. 2017. “Mask r-cnn.” In Proc., IEEE Int. Conf., 2961–2969. New York: IEEE.
Huang, J., Y. Fang, Y. Wu, H. Wu, Z. Gao, Y. Li, J. D. Ser, J. Xia, and G. Yang. 2022. “Swin transformer for fast MRI.” Neurocomputing 493 (Apr): 281–304. https://doi.org/10.1016/j.neucom.2022.04.051.
Khan, S., M. Naseer, M. Hayat, S. W. Zamir, F. S. Khan, and M. Shah. 2022. “Transformers in vision: A survey.” ACM Comput. Surv. 54 (10s): 1–41. https://doi.org/10.1145/3505244.
Lin, T. Y., P. Goyal, R. Girshick, K. He, and P. Dollár. 2017. “Focal loss for dense object detection.” In Proc., IEEE Int. Conf. on Computer Vision, 2980–2988. New York: IEEE.
Lin, T. Y., M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. 2014. “Microsoft coco: Common objects in context.” In Proc., Computer Vision–ECCV 2014: 13th European Conf., 740–755. New York: Springer.
Linton, M. A., and L. Fu. 2016. “Connected vehicle solution for winter road surface condition monitoring.” Transp. Res. Rec. 2551 (1): 62–72. https://doi.org/10.3141/2551-08.
Liu, S., L. Qi, H. Qin, J. Shi, and J. Jia. 2018. “Path aggregation network for instance segmentation.” In Proc., 2018 IEEE Computer Vision and Pattern Recognition Conf., 8759–8768. New York: IEEE.
Liu, Z., Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo. 2021. “Swin transformer: Hierarchical vision transformer using shifted windows.” In Proc., 2021  IEEE/CVF Int. Conf. on Computer Vision, 10012–10022. New York: IEEE.
Pan, G., M. Muresan, R. Yu, and L. Fu. 2021. “Real-time winter road surface condition monitoring using an improved residual CNN.” Can. J. Civ. Eng. 48 (9): 1215–1222. https://doi.org/10.1139/cjce-2019-0367.
Rahman, F. U., M. T. Ahmed, M. R. Amin, N. Nabi, and M. M. S. Ahamed. 2022, “A comparative study on road surface state assessment using transfer learning approach.” In Proc., 13th Int. Conf. on Computing Communication and Networking Technologies, 1–6. New York: IEEE.
Ren, S., K. He, R. Girshick, and J. Sun. 2015. “Faster R-CNN: Towards real-time object detection with region proposal networks.” IEEE Trans. Pattern Anal. Mach. Intell. 39 (6): 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031.
Sun, P., et al. 2021. “Sparse R-CNN: End-to-end object detection with learnable proposals.” In Proc., IEEE/CVF Computer Vision and Pattern Recognition Conf., 14454–14463. New York: IEEE.
Vachmanus, S., A. A. Ravankar, T. Emaru, and Y. Kobayashi. 2020. “Semantic segmentation for road surface detection in snowy environment.” In Proc., 2020 59th Annual Conf. of the Society of Instrument and Control Engineers of Japan (SICE), 1381–1386. New York: IEEE.
Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. 2017. “Attention is all you need.” In Proc., 2017 31st Advances in Neural Information Processing Systems, 30. Glasgow, Scotland: Curran Associates.
Wang, N., Y. Wang, and M. J. Er. 2022. “Review on deep learning techniques for marine object recognition: Architectures and algorithms.” Control Eng. Pract. 118 (Jan): 104458. https://doi.org/10.1016/j.conengprac.2020.104458.
Wang, Y., D. Zhang, Y. Liu, B. Dai, and L. H. Lee. 2019. “Enhancing transportation systems via deep learning: A survey.” Transp. Res. Part C Emerging Technol. 99 (Feb): 144–163. https://doi.org/10.1016/j.trc.2018.12.004.
Xie, Q., and T. J. Kwon. 2022. “Development of a highly transferable urban winter road surface classification model: A deep learning approach.” Transp. Res. Rec. 2676 (10): 445–459. https://doi.org/10.1177/03611981221090235.
Zhang, H., H. Chang, B. Ma, N. Wang, and X. Chen. 2020. “Dynamic R-CNN: Towards high quality object detection via dynamic training.” In Proc., Computer Vision–ECCV 2020: 16th European Conf., 260–275. New York: Springer.
Zhang, Z., M. Li, X. Lin, Y. Wang, and F. He. 2019. “Multistep speed prediction on Traffic Networks: A deep learning approach considering spatio-temporal dependencies.” Transp. Res. Part C Emerging Technol. 105 (Apr): 297–322. https://doi.org/10.1016/j.trc.2019.05.039.

Information & Authors

Information

Published In

Go to Journal of Transportation Engineering, Part B: Pavements
Journal of Transportation Engineering, Part B: Pavements
Volume 150Issue 3September 2024

History

Received: Aug 25, 2023
Accepted: Feb 27, 2024
Published online: May 16, 2024
Published in print: Sep 1, 2024
Discussion open until: Oct 16, 2024

Permissions

Request permissions for this article.

ASCE Technical Topics:

Authors

Affiliations

Zhiyuan Bai
Master’s Student, Dept. of Automatic and Electrical Engineering, Linyi Univ., North Side of the Middle Section of Shuangling Rd., Linyi City, Shandong 276000, China.
Yue Wang
Master’s Student, Dept. of Automatic and Electrical Engineering, Linyi Univ., North Side of the Middle Section of Shuangling Rd., Linyi City, Shandong 276000, China.
Ancai Zhang
Professor, Dept. of Automatic and Electrical Engineering, Linyi Univ., North Side of the Middle Section of Shuangling Rd., Linyi City, Shandong 276000, China.
Hao Wei
Lecturer, Dept. of Automatic and Electrical Engineering, Linyi Univ., North Side of the Middle Section of Shuangling Rd., Linyi City, Shandong 276000, China.
Associate Professor, Dept. of Automatic and Electrical Engineering, Linyi Univ., North Side of the Middle Section of Shuangling Rd., Linyi City, Shandong 276000, China (corresponding author). ORCID: https://orcid.org/0000-0003-0115-6659. Email: [email protected]

Metrics & Citations

Metrics

Citations

Download citation

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

View Options

Get Access

Access content

Please select your options to get access

Log in/Register Log in via your institution (Shibboleth)
ASCE Members: Please log in to see member pricing

Purchase

Save for later Information on ASCE Library Cards
ASCE Library Cards let you download journal articles, proceedings papers, and available book chapters across the entire ASCE Library platform. ASCE Library Cards remain active for 24 months or until all downloads are used. Note: This content will be debited as one download at time of checkout.

Terms of Use: ASCE Library Cards are for individual, personal use only. Reselling, republishing, or forwarding the materials to libraries or reading rooms is prohibited.
ASCE Library Card (5 downloads)
$105.00
Add to cart
ASCE Library Card (20 downloads)
$280.00
Add to cart
Buy Single Article
$35.00
Add to cart

Get Access

Access content

Please select your options to get access

Log in/Register Log in via your institution (Shibboleth)
ASCE Members: Please log in to see member pricing

Purchase

Save for later Information on ASCE Library Cards
ASCE Library Cards let you download journal articles, proceedings papers, and available book chapters across the entire ASCE Library platform. ASCE Library Cards remain active for 24 months or until all downloads are used. Note: This content will be debited as one download at time of checkout.

Terms of Use: ASCE Library Cards are for individual, personal use only. Reselling, republishing, or forwarding the materials to libraries or reading rooms is prohibited.
ASCE Library Card (5 downloads)
$105.00
Add to cart
ASCE Library Card (20 downloads)
$280.00
Add to cart
Buy Single Article
$35.00
Add to cart

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share