A Novel Filtering Method of Travel-Time Outliers Extracted from Large-Scale Traffic Checkpoint Data
Publication: Journal of Transportation Engineering, Part A: Systems
Volume 150, Issue 2
Abstract
The traffic checkpoint data collected by video detection and license plate recognition technologies contain abundant traffic information such as vehicle travel time and path. However, due to various factors, there are often many outliers in the travel-time data, which may lead to inaccurate research results. How to extract effective travel-time data from the traffic checkpoint data is one of the key questions for urban traffic evolution. Based on the large-scale checkpoint data from a city in Guangdong Province, China, this study proposes a statistics-based filtering algorithm for travel-time outliers and applies it to urban vehicle travel path and origin-destination (OD) identification. The sampling rate and reading rate of checkpoint data are verified. The sampling rate and reading rate are 99.46% and 93.7%, respectively, and the reliability of the data is high. A statistics-based filtering algorithm for travel-time outliers is proposed and compared with three existing clustering algorithms. The results showed that for a load section of 376.54 m, the maximum travel time is reached during the period 18:00–18:15. The average travel time is 5.55 min, and the maximum travel time is 8.08 min, which is consistent with the actual operation of the road section. The travel time obtained by the three clustering methods does not reflect the actual operation of the road section well. This means that the effective travel time extracted by this method is more practical and can reflect the fluctuation of travel time on urban roads. Then, the sensitivity of the proposed method to the time window and road length is tested. The results showed that the time window setting will affect the outlier-filtering effect, and the road length has less effect on the proposed method. Finally, the proposed algorithm is applied to single travel identification of urban vehicles to extract single travel paths and OD information of vehicles, which can provide accurate data for urban traffic research.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all of the data, models, or code supporting the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (No. 52072131), the Key Research Projects of Universities in Guangdong Province (No. 2019KZDXM009), and the Natural Science Foundation of Guangdong Province (No. 2023A1515010039).
References
Ahmed, M., R. Seraj, and S. Islam. 2020. “The -means algorithm: A comprehensive survey and performance evaluation.” Electronics 9 (8): 1295. https://doi.org/10.3390/electronics9081295.
Ali, B., N. Azam, A. Shah, and J. Yao. 2021. “A spatial filtering inspired three-way clustering approach with application to outlier detection.” Int. J. Approximate Reasoning 130 (Mar): 1–21. https://doi.org/10.1016/j.ijar.2020.12.003.
Bagirov, A. M., R. M. Aliguliyev, and N. Sultanova. 2023. “Finding compact and well-separated clusters: Clustering using silhouette coefficients.” Pattern Recognit. 135 (Mar): 109144. https://doi.org/10.1016/j.patcog.2022.109144.
Chakraborty, B., A. Chaterjee, S. Malakar, and R. Sarkar. 2022. “An iterative approach to unsupervised outlier detection using ensemble method and distance-based data filtering.” Complex Intell. Syst. 8 (4): 3215–3230. https://doi.org/10.1007/s40747-022-00674-0.
Chen, S., W. Wang, and H. van Zuylen. 2010. “A comparison of outlier detection algorithms for its data.” Expert Syst. Appl. 37 (2): 1169–1178. https://doi.org/10.1016/j.eswa.2009.06.008.
Dixon, M. P., and L. R. Rilett. 2005. “Population origin-destination estimation using automatic vehicle identification and volume data.” J. Transp. Eng. 131 (2): 75–82. https://doi.org/10.1061/(ASCE)0733-947X(2005)131:2(75).
El Esawey, M., and T. Sayed. 2012. “A framework for neighbour links travel time estimation in an urban network.” Transp. Plann. Technol. 35 (3): 281–301. https://doi.org/10.1080/03081060.2012.671028.
Fränti, P., and S. Sieranoja. 2018. “K-means properties on six clustering benchmark datasets.” Appl. Intell. 48 (12): 4743–4759. https://doi.org/10.1007/s10489-018-1238-7.
Fu, C., and T. Sayed. 2023. “Identification of adequate sample size for conflict-based crash risk evaluation: An investigation using bayesian hierarchical extreme value theory models.” Anal. Methods Accid. Res. 39 (Sep): 100281. https://doi.org/10.1016/j.amar.2023.100281.
Fu, F. J., W. Qian, and H. Z. Dong. 2019. “Estimation of route travel time distribution with information fusion from automatic number plate recognition data.” J. Transp. Eng. Part A. Syst. 145 (7): 0401–9029. https://doi.org/10.1061/JTEPBS.0000242.
Guo, J. H., C. G. Li, X. Qin, W. Huang, Y. Wei, and J. D. Cao. 2019. “Analyzing distributions for travel time data collected using radio frequency identification technique in urban road networks.” Sci. China-Technol. Sci. 62 (1): 106–120. https://doi.org/10.1007/s11431-018-9267-4.
Hahsler, M., M. Piekenbrock, and D. Doran. 2019. “Dbscan: Fast density-based clustering with R.” J. Stat. Software 91 (1): 1–30. https://doi.org/10.18637/jss.v091.i01.
Jang, J. 2016. “Outlier filtering algorithm for travel time estimation using dedicated short-range communications probes on rural highways.” IET Intel. Transport Syst. 10 (6): 453–460. https://doi.org/10.1049/iet-its.2015.0103.
Jayakumar, K., and M. G. Babu. 2018. “Discrete Weibull geometric distribution and its properties.” Commun. Stat.- Theory Methods 47 (7): 1767–1783. https://doi.org/10.1080/03610926.2017.1327074.
Jin, X. L., R. C. Tang, L. F. Liu, and J. G. Wu. 2021. “Vehicle license plate recognition for fog-haze environments.” IET Image Proc. 15 (6): 1273–1284. https://doi.org/10.1049/ipr2.12103.
Kieu, L. M., A. Bhaskar, and E. Chung. 2015. “Public transport travel-time variability definitions and monitoring.” J. Transp. Eng. 141 (1): 04014068. https://doi.org/10.1061/(ASCE)TE.1943-5436.0000724.
Kim, J., and H. S. Mahmassani. 2014. “A finite mixture model of vehicle-to-vehicle and day-to-day variability of traffic network travel times.” Transp. Res. Part C Emerging Technol. 46 (Sep): 83–97. https://doi.org/10.1016/j.trc.2014.05.011.
Lara, J. A., D. Lizcano, V. Ramperez, and J. Soriano. 2020. “A method for outlier detection based on cluster analysis and visual expert criteria.” Expert Syst. 37 (5): e12473. https://doi.org/10.1111/exsy.12473.
Li, G., M. R. Mahmoudi, S. N. Qasem, B. A. Tuan, and K. H. Pho. 2020. “Cluster ensemble of valid small clusters.” J. Intell. Fuzzy Syst. 39 (1): 525–542. https://doi.org/10.3233/JIFS-191530.
Lin, G., L. Xin, H. Feng, and L. Ying. 2014. “A new outlier detection algorithm and its application in intelligent transportation system.” In Proc., 2014 IEEE 7th Joint Int. Information Technology and Artificial Intelligence Conf., 442–445. New York: IEEE.
Ma, Z., and P. Zhang. 2022. “Individual mobility prediction review: Data, problem, method and application.” Multimodal Transp. 1 (1): 100002. https://doi.org/10.1016/j.multra.2022.100002.
Mo, B. C., R. M. Li, and J. C. Dai. 2020. “Estimating dynamic origin-destination demand: A hybrid framework using license plate recognition data.” Comput.-Aided Civ. Infrastruct. Eng. 35 (7): 734–752. https://doi.org/10.1111/mice.12526.
Moghaddam, S. S., and B. Hellinga. 2013. “Evaluating the performance of algorithms for the detection of travel time outliers.” Transp. Res. Rec. 2338 (1): 67–77. https://doi.org/10.3141/2338-08.
Murtagh, F., and P. Contreras. 2017. “Algorithms for hierarchical clustering: An overview, II.” Wiley Interdiscip. Rev.: Data Min. Knowl. Discovery 7 (6): e1219. https://doi.org/10.1002/widm.1219.
Nielsen, F. 2020. “On a generalization of the Jensen-Shannon divergence and the Jensen-Shannon centroid.” Entropy 22 (2): 221. https://doi.org/10.3390/e22020221.
Park, H., and Y. Kim. 2018. “Model for filtering the outliers in DSRC travel time data on interrupted traffic flow sections.” KSCE J. Civ. Eng. 22 (9): 3607–3619. https://doi.org/10.1007/s12205-017-1333-z.
Park, J., J. Lee, I. J. Kim, and K. Sohn. 2022. “Probabilistic representations for video contrastive learning.” In Proc., 2022 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 14691–14701. New York: IEEE.
Pulugurtha, S. S., and R. N. Jain. 2022. “Passenger car equivalent travel time of a truck.” Multimodal Transp. 1 (3): 100031. https://doi.org/10.1016/j.multra.2022.100031.
Rao, W. M., Y. J. Wu, J. X. Xia, J. S. Ou, and R. Kluger. 2018. “Origin-destination pattern estimation based on trajectory reconstruction using automatic license plate recognition data.” Transp. Res. Part C Emerging Technol. 95 (Jan): 29–46. https://doi.org/10.1016/j.trc.2018.07.002.
Shi, C. Y., B. Y. Chen, and Q. Q. Li. 2017. “Estimation of travel time distributions in urban road networks using low-frequency floating car data.” ISPRS Int. J. Geo-Inf. 6 (8): 253. https://doi.org/10.3390/ijgi6080253.
Tang, K., S. Chen, Z. Liu, and A. J. Khattak. 2018. “A tensor-based Bayesian probabilistic model for citywide personalized travel time estimation.” Transp. Res. Part C Emerging Technol. 90 (May): 260–280. https://doi.org/10.1016/j.trc.2018.03.004.
Van Erven, T., and P. Harremoes. 2014. “Rényi divergence and Kullback-Leibler divergence.” IEEE Trans. Inf. Theory 60 (7): 3797–3820. https://doi.org/10.1109/TIT.2014.2320500.
Williams, J., R. R. Hill, J. J. Pignatiello, and E. Chicken. 2022. “Wavelet analysis of variance box plot.” J. Appl. Stat. 49 (14): 3536–3563. https://doi.org/10.1080/02664763.2021.1951685.
Wu, Z. F., Z. X. Wu, and L. R. Rilett. 2020. “Innovative nonparametric method for data outlier filtering.” Transp. Res. Rec. 2674 (10): 167–176. https://doi.org/10.1177/0361198120945697.
Xie, H. H., Z. Yu, Z. C. He, K. Y. Chen, J. P. Liu, and W. M. Xie. 2023. “Analyzing overnight parking demand characteristics using city-scale automatic vehicle identification data in shanghai, china.” J. Transp. Eng. Part A. Syst. 149 (1): 0402–2121. https://doi.org/10.1061/JTEPBS.TEENG-7254.
Yang, G. C., D. Coble, C. Vaughan, C. Peele, A. Morsali, G. F. List, and D. J. Findley. 2022. “Waiting time estimation at ferry terminals based on license plate recognition.” J. Transp. Eng. Part A. Syst. 148 (9): 0402–2064. https://doi.org/10.1061/JTEPBS.0000722.
Yuan, S., B. Wright, Y. Zou, and Y. Wang. 2017. “Quantification of variability of valid travel times with FMMS for buses, passenger cars, and taxis.” IET Intel. Transport Syst. 11 (1): 1–9. https://doi.org/10.1049/iet-its.2016.0017.
Zhao, H., W. Gunardi, Y. Liu, C. Kiew, T. H. Teng, and X. B. Yang. 2022. “Prediction of traffic incident duration using clustering-based ensemble learning method.” J. Transp. Eng. Part A. Syst. 148 (7): 0402–2044. https://doi.org/10.1061/JTEPBS.0000688.
Information & Authors
Information
Published In
Journal of Transportation Engineering, Part A: Systems
Volume 150 • Issue 2 • February 2024
Copyright
© 2023 American Society of Civil Engineers.
History
Received: May 10, 2023
Accepted: Aug 28, 2023
Published online: Nov 22, 2023
Published in print: Feb 1, 2024
Discussion open until: Apr 22, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.