Traffic State Prediction for Urban Networks: A Spatial–Temporal Transformer Network Model
Publication: Journal of Transportation Engineering, Part A: Systems
Volume 149, Issue 11
Abstract
Traffic state prediction plays an important role in traffic management, e.g., it can provide travelers with accurate routing information to achieve a better travel experience. In this paper, we propose a spatial-temporal transformer network (STTN) model on the traffic state prediction for ubran networks. The STTN model integrates four modules: road embedding (RE); basic information embedding (BIE); temporal transformer (TT); and spatial-temporal transformer (STT). Specifically, the road topology information and other basic road information are embedded in the RE and BIE modules, respectively. The TT module, which is developed based on the Transformer encoder, captures the variation of the sequential historical traffic flow data. The STT module fuses a TT, which captures the spatial correlations and temporal dynamics of network traffic state, and the attention mechanism, which adjusts the importance of different historical data. The performance of the proposed STTN model is demonstrated using real traffic data collected from crowd-sourced vehicles. The proposed model achieves better prediction accuracy in terms of f1-score and weighted f1-score compared with those of other baseline models. The ablation study shows that some modules in the proposed STTN have a significant impact on improving short-term prediction ability.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some data and code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This research was sponsored by the National Key R&D Program of China (2022ZD0115600), Natural Science Foundation of Jiangsu (No. BK20200378), the Postgraduate Research & Practice Innovation Program of Jiangsu Province (Grant No. KYCX22_0279), and the National Natural Science Foundation of China (No. 52131203).
References
Afrin, T., and N. Yodo. 2022. “A long short-term memory-based correlated traffic data prediction framework.” Knowledge-Based Syst. 237 (Feb): 107755. https://doi.org/10.1016/j.knosys.2021.107755.
BDCI (Big Data & Computing Intelligence Contest). 2020. “CCF BDCI.” Accessed October 20, 2020. https://www.datafountain.cn/competitions/466.
Bhanu, M., J. Mendes-Moreira, and J. Chandra. 2020. “Embedding traffic network characteristics using tensor for improved traffic prediction.” IEEE Trans. Intell. Transp. Syst. 22 (6): 3359–3371. https://doi.org/10.1109/TITS.2020.2984175.
Cai, L., K. Janowicz, G. Mai, B. Yan, and R. Zhu. 2020. “Traffic transformer: Capturing the continuity and periodicity of time series for traffic forecasting.” Trans. GIS 24 (3): 736–755. https://doi.org/10.1111/tgis.12644.
Chen, Q., Z. Wei, X. Wang, L. Li, and Y. Lv. 2022. “Social relation and physical lane aggregator: Integrating social and physical features for multimodal motion prediction.” J. Intell. Connected Veh. 5 (3): 302–308. https://doi.org/10.1108/JICV-07-2022-0028.
Chen, T., and C. Guestrin. 2016. “Xgboost: A scalable tree boosting system.” In Proc., of the 22nd ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, KDD’16, 785–794. New York: Association for Computing Machinery. https://doi.org/10.1145/2939672.2939785.
Cortes, C., and V. Vapnik. 1995. “Support-vector networks.” Mach. Learn. 20 (3): 273–297. https://doi.org/10.1007/BF00994018.
Devlin, J., M.-W. Chang, K. Lee, and K. Toutanova. 2018. “Bert: Pre-training of deep bidirectional transformers for language understanding.” Preprint, submitted September 26, 2022. https://arxiv.org/abs/1810.04805.
Dong, J., S. Chen, M. Miralinaghi, T. Chen, and S. Labi. 2022. “Development and testing of an image transformer for explainable autonomous driving systems.” J. Intell. Connected Veh. 5 (3): 235–249. https://doi.org/10.1108/JICV-06-2022-0021.
Dosovitskiy, A., et al. 2020. “An image is worth 16×16 words: Transformers for image recognition at scale.” Preprint, submitted June 23, 2021. https://arxiv.org/abs/2010.11929.
Du, M., L. Yang, Y. Jin, and J. Tu. 2021. “Vehicle global path planning algorithm based on spatio-temporal characteristics of traffic.” J. Automot. Saf. Energy 12 (1): 52.
Fang, L., Z. Guan, T. Wang, J. Gong, and F. Du. 2022. “Collision avoidance model and its validation for intelligent vehicles based on deep learning LSTM.” J. Automot. Saf. Energy 13 (1): 104.
Ge, W., Y. Cao, Z. Ding, and L. Guo. 2019. “Forecasting model of traffic flow prediction model based on multi-resolution SVR.” In Proc., 2019 3rd Int. Conf. on Innovation in Artificial Intelligence, 1–5. New York: Association for Computing Machinery. https://doi.org/10.1145/3319921.3319923.
Girshick, R. 2015. “Fast r-CNN.” In Proc., IEEE Int. Conf. on Computer Vision, 1440–1448. New York: IEEE.
Guan, D., L. Huang, and Q. Qu. 2018. “A predicting method of urban traffic network volume based on STARIMA model.” In Proc., 17th CICTP 2017: Transportation Reform and Change—Equity, Inclusiveness, Sharing, and Innovation, 3600–3606. Reston, VA: ASCE.
Guillot, M., A. Furno, E.-H. Aghezzaf, and N.-E. El Faouzi. 2022. “Transport network downsizing based on optimal sub-network.” Commun. Transp. Res. 2 (Dec): 100079. https://doi.org/10.1016/j.commtr.2022.100079.
Guo, S., Y. Lin, N. Feng, C. Song, and H. Wan. 2019. “Attention based spatial-temporal graph convolutional networks for traffic flow forecasting.” In Vol. 33 of Proc., AAAI Conf. on Artificial Intelligence, 922–929. Washington, DC: Association for the Advancement of Artificial Intelligence.
Hamed, M. M., H. R. Al-Masaeid, and Z. M. B. Said. 1995. “Short-term prediction of traffic volume in urban arterials.” J. Transp. Eng. 121 (3): 249–254. https://doi.org/10.1061/(ASCE)0733-947X(1995)121:3(249).
Hu, Y., T. Jiang, X. Liu, and Y. Shi. 2022. “Pedestrian-crossing intention-recognition based on dual-stream adaptive graph-convolutional neural-network.” J. Automot. Saf. Energy 13 (2): 325.
Jo, D., B. Yu, H. Jeon, and K. Sohn. 2018. “Image-to-image learning to predict traffic speeds by considering area-wide spatio-temporal dependencies.” IEEE Trans. Veh. Technol. 68 (2): 1188–1197. https://doi.org/10.1109/TVT.2018.2885366.
Kingma, D. P., and B. J. Adam. 2018. “A method for stochastic optimization.” Preprint, submitted March 8, 2023. http://arxiv.org/abs/1412.6980.
Lecun, Y., Y. Bengio, and G. Hinton. 2015. “Deep learning.” Nature 521 (7553): 436. https://doi.org/10.1038/nature14539.
Li, Z., G. Xiong, Y. Chen, Y. Lv, B. Hu, F. Zhu, and F.-Y. Wang. 2019. “A hybrid deep learning approach with GCN and LSTM for traffic flow prediction.” In Proc., IEEE Intelligent Transportation Systems Conf. (ITSC), 1929–1933. New York: IEEE.
Lin, T.-Y., P. Goyal, R. Girshick, K. He, and P. Dollár. 2017. “Focal loss for dense object detection.” In Proc., IEEE Int. Conf. on Computer Vision, 2980–2988. New York: IEEE.
Liu, Y., et al. 2023. “Reliability of electric vehicle charging infrastructure: A cross-lingual deep learning approach.” Commun. Transp. Res. 3 (Dec): 100095. https://doi.org/10.1016/j.commtr.2023.100095.
Liu, Y., R. Jia, J. Ye, and X. Qu. 2022. “How machine learning informs ride-hailing services: A survey.” Commun. Transp. Res. 2 (Dec): 100075. https://doi.org/10.1016/j.commtr.2022.100075.
Liu, Y., Z. Liu, H. L. Vu, and C. Lyu. 2020. “A spatio-temporal ensemble method for large-scale traffic state prediction.” Comput.-Aided Civ. Infrastruct. Eng. 35 (1): 26–44. https://doi.org/10.1111/mice.12459.
Liu, Y., C. Lyu, A. Khadka, W. Zhang, and Z. Liu. 2019. “Spatio-temporal ensemble method for car-hailing demand prediction.” IEEE Trans. Intell. Transp. Syst. 21 (12): 5328–5333. https://doi.org/10.1109/TITS.2019.2948790.
Liu, Y., C. Lyu, Y. Zhang, Z. Liu, W. Yu, and X. Qu. 2021a. “Deeptsp: Deep traffic state prediction model based on large-scale empirical data.” Commun. Transp. Res. 1 (Dec): 100012. https://doi.org/10.1016/j.commtr.2021.100012.
Liu, Y., F. Wu, C. Lyu, X. Liu, and Z. Liu. 2021b. “Behavior2vector: Embedding users’ personalized travel behavior to vector.” IEEE Trans. Intell. Transp. Syst. 23 (7): 8346–8355. https://doi.org/10.1109/TITS.2021.3078229.
Mikolov, T., I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. 2013. “Distributed representations of words and phrases and their compositionality.” In Vol. 26 of Advances in neural information processing systems. Red Hook, NY: Curran Associates.
Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. 2017. “Attention is all you need.” In Vol. 26 of Advances in neural information processing systems, 5998–6008. Red Hook, NY: Curran Associates.
Vlahogianni, E. I., M. G. Karlaftis, and J. C. Golias. 2014. “Short-term traffic forecasting: Where we are and where we’re going.” Transp. Res. Part C Emerging Technol. 43 (Jun): 3–19. https://doi.org/10.1016/j.trc.2014.01.005.
Williams, B. M., and L. A. Hoel. 2003. “Modeling and forecasting vehicular traffic flow as a seasonal ARIMA process: Theoretical basis and empirical results.” J. Transp. Eng. 129 (6): 664–672. https://doi.org/10.1061/(ASCE)0733-947X(2003)129:6(664).
Xie, Y., Y. Zhang, and Z. Ye. 2007. “Short-term traffic volume forecasting using Kalman filter with discrete wavelet decomposition.” Comput.-Aided Civ. Infrastruct. Eng. 22 (5): 326–334. https://doi.org/10.1111/j.1467-8667.2007.00489.x.
Yan, H., L. Fu, Y. Qi, L. Cheng, Q. Ye, and D.-J. Yu. 2022. “Learning a robust classifier for short-term traffic state prediction.” Knowledge-Based Syst. 242 (Apr): 108368. https://doi.org/10.1016/j.knosys.2022.108368.
Yan, H., X. Ma, and Z. Pu. 2021. “Learning dynamic and hierarchical traffic spatiotemporal features with transformer.” IEEE Trans. Intell. Transp. Syst. 23 (11): 22386–22399. https://doi.org/10.1109/TITS.2021.3102983.
Yu, B., X. Song, F. Guan, Z. Yang, and B. Yao. 2016. “k-nearest neighbor model for multiple-time-step prediction of short-term traffic condition.” J. Transp. Eng. 142 (6): 04016018. https://doi.org/10.1061/(ASCE)TE.1943-5436.0000816.
Yu, R., Y. Li, C. Shahabi, U. Demiryurek, and Y. Liu. 2017. “Deep learning: A generic approach for extreme condition traffic forecasting.” In Proc., 2017 SIAM Int. Conf. on Data Mining, 777–785. Philadelphia: Society for Industrial and Applied Mathematics.
Zhang, J., Y. Zheng, and D. Qi. 2017. “Deep spatio-temporal residual networks for citywide crowd flows prediction.” In Proc., 31st AAAI Conf. on Artificial Intelligence. Washington, DC: Association for the Advancement of Artificial Intelligence.
Zhang, J., Y. Zheng, D. Qi, R. Li, and X. Yi. 2016. “DNN-based prediction model for spatio-temporal data.” In Proc., 24th ACM SIGSPATIAL Int. Conf. on Advances in Geographic Information Systems, 1–4. New York: Association for Computing Machinery. https://doi.org/10.1145/2996913.2997016.
Zhang, S., Y. Chen, and W. Zhang. 2021. “Spatiotemporal fuzzy-graph convolutional network model with dynamic feature encoding for traffic forecasting.” Knowledge-Based Syst. 231 (Nov): 107403. https://doi.org/10.1016/j.knosys.2021.107403.
Zhang, W., K. Zhu, S. Zhang, Q. Chen, and J. Xu. 2022. “Dynamic graph convolutional networks based on spatiotemporal data embedding for traffic flow forecasting.” Knowledge-Based Syst. 250 (Aug): 109028. https://doi.org/10.1016/j.knosys.2022.109028.
Zhang, Y., T. Cheng, and Y. Ren. 2019a. “A graph deep learning method for short-term traffic forecasting on large road networks.” Comput.-Aided Civ. Infrastruct. Eng. 34 (10): 877–896. https://doi.org/10.1111/mice.12450.
Zhang, Z., M. Li, X. Lin, Y. Wang, and F. He. 2019b. “Multistep speed prediction on traffic networks: A deep learning approach considering spatio-temporal dependencies.” Transp. Res. Part C Emerging Technol. 105 (Aug): 297–322. https://doi.org/10.1016/j.trc.2019.05.039.
Zhao, L., Y. Song, C. Zhang, Y. Liu, P. Wang, T. Lin, M. Deng, and H. Li. 2019. “T-GCN: A temporal graph convolutional network for traffic prediction.” IEEE Trans. Intell. Transp. Syst. 21 (9): 3848–3858. https://doi.org/10.1109/TITS.2019.2935152.
Zheng, Z., and D. Su. 2014. “Short-term traffic volume forecasting: A k-nearest neighbor approach enhanced by constrained linearly sewing principle component algorithm.” Transp. Res. Part C Emerging Technol. 43 (Jun): 143–157. https://doi.org/10.1016/j.trc.2014.02.009.
Zhong, C., P. Wu, Q. Zhang, and Z. Ma. 2023. “Online prediction of network-level public transport demand based on principle component analysis.” Commun. Transp. Res. 3 (Dec): 100093. https://doi.org/10.1016/j.commtr.2023.100093.
Zhu, W., J. Wu, T. Fu, J. Wang, J. Zhang, and Q. Shangguan. 2021. “Dynamic prediction of traffic incident duration on urban expressways: A deep learning approach based on LSTM and MLP.” J. Intell. Connected Veh. 4 (2): 80–91. https://doi.org/10.1108/JICV-03-2021-0004.
Information & Authors
Information
Published In
Journal of Transportation Engineering, Part A: Systems
Volume 149 • Issue 11 • November 2023
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Dec 19, 2022
Accepted: Jun 23, 2023
Published online: Aug 25, 2023
Published in print: Nov 1, 2023
Discussion open until: Jan 25, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.