Validation of Machine Learning Algorithms as Predictive Tool in the Road Safety Management Process: Case of Network Screening
Publication: Journal of Transportation Engineering, Part A: Systems
Volume 148, Issue 9
Abstract
Safety performance functions (SPFs) are the key regression tools in the road safety management process (RSMP) and are used to predict crash frequency given a set of roadway and traffic factors. Although regression-based SPFs have been proven to be reliable tools for road safety predictive analytics, some limitations and constrains have been highlighted in the literature, such as the need to assume a probability distribution, the need to select a predefined functional form, possible correlation between independent variables, and possible transferability issues. An alternative to traditional regression models as predictive tools is the use of machine learning (ML) algorithms. This research compared the prediction performance of three well-known ML algorithms, i.e., support vector machine (SVM), decision tree (DT), and random forest (RF), with that of traditional SPFs, and applied and validated ML algorithms in network screening, which is the first step in the RSMP. To achieve these objectives, traditional SPFs using negative-binomial (NB) generalized linear regression were estimated and compared with ML algorithms using three different goodness-of -fit criteria. A data set of urban signalized and unsignalized intersections from two major municipalities in Saskatchewan (Canada) was considered as a case study. Ranking consistency tests of collision-prone locations identified using ML-based and SPF-based performance measures were conducted. The results showed that the consistency of ML-based measures in identifying hotspots was comparable to that of SPF-based measures, particularly the excess (predicted and expected) average crash frequency. Overall, the results of this research support the use of SVM, DT, and RF as predictive tools in network screening.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This research was supported by a discovery grant from the Natural Sciences and Engineering Research Council of Canada (NSERC) (RGPIN 05288-2017).
References
AASHTO. 2010. Highway safety manual. Washington, DC: AASHTO.
Abdel-Aty, M., and K. Haleem. 2011. “Analyzing angle crashes at unsignalized intersections using machine learning techniques.” Accid. Anal. Prev. 43 (1): 461–470. https://doi.org/10.1016/j.aap.2010.10.002.
Abdelwahab, H. T., and M. A. Abdel-Aty. 2002. “Artificial neural networks and logit models for traffic safety analysis of toll plazas.” Transp. Res. Rec. 1784 (1): 115–125. https://doi.org/10.3141/1784-15.
Chang, L.-Y. 2005. “Analysis of freeway accident frequencies: Negative binomial regression versus artificial neural network.” Saf. Sci. 43 (8): 541–557. https://doi.org/10.1016/j.ssci.2005.04.004.
Chen, C., G. Zhang, Z. Qian, R. Tarefder, and Z. Tian. 2016. “Investigating driver injury severity patterns in rollover crashes using support vector machine models.” Accid. Anal. Prev. 90 (May): 128–139. https://doi.org/10.1016/j.aap.2016.02.011.
Cheng, W., and S. Washington. 2008. “New criteria for evaluating methods of identifying hot spots.” Transp. Res. Rec. 2083 (1): 76–85. https://doi.org/10.3141/2083-09.
Dong, C., C. Shao, and J. Li. 2015. “An improved deep learning model for traffic crash prediction.” J. Adv. Transp.
Dong, N., H. Huang, and L. Zheng. 2015. “Support vector machine in crash prediction at the level of traffic analysis zones: Assessing the spatial proximity effects.” Accid. Anal. Prev. 82 (Sep): 192–198. https://doi.org/10.1016/j.aap.2015.05.018.
El-Basyouny, K., and T. Sayed. 2009. “Collision prediction models using multivariate Poisson-lognormal regression.” Accid. Anal. Prev. 41 (4): 820–828. https://doi.org/10.1016/j.aap.2009.04.005.
Geedipally, S. R., and D. Lord. 2010. “Investigating the effect of modeling single-vehicle and multi-vehicle crashes separately on confidence intervals of Poisson–gamma models.” Accid. Anal. Prev. 42 (4): 1273–1282. https://doi.org/10.1016/j.aap.2010.02.004.
Géron, A. 2017. Hands-on machine learning with scikit-learn and tensorflow: Concepts, tools, and techniques to build intelligent systems. 1st ed. Newton, MA: O’Reilly Media.
Hauer, E. 1997. Observational before–after studies in road safety. Oxford, UK: Elsevier Science.
Hauer, E. 2015. Vol. 38 of The art of regression modeling in road safety. New York: Springer.
Huang, H., Q. Zeng, X. Pei, S. C. Wong, and P. Xu. 2016. “Predicting crash frequency using an optimised radial basis function neural network model.” Transportmetrica Transp. Sci. 12 (4): 330–345. https://doi.org/10.1080/23249935.2015.1136008.
Kim, D.-G., Y. Lee, S. Washington, and K. Choi. 2007. “Modeling crash outcome probabilities at rural intersections: Application of hierarchical binomial logistic models.” Accid. Anal. Prev. 39 (1): 125–134. https://doi.org/10.1016/j.aap.2006.06.011.
Lan, B., and B. Persaud. 2011. “Fully Bayesian approach to investigate and evaluate ranking criteria for black spot identification.” Transp. Res. Rec. 2237 (1): 117–125. https://doi.org/10.3141/2237-13.
Lie, X., D. Lord, Y. Zhang, and Y. Xie. 2008. “Predicting motor vehicle crashes using support vector machine models.” Accid. Anal. Prev. 40 (4): 1611–1618. https://doi.org/10.1016/j.aap.2008.04.010.
Lord, D., and F. Mannering. 2010. “The statistical analysis of crash-frequency data: A review and assessment of methodological alternatives.” Transp. Res. Part A Policy Pract. 44 (5): 291–305. https://doi.org/10.1016/j.tra.2010.02.001.
Marsland, S. 2009. Machine learning: An algorithmic perspective. 2nd ed. New York: CRC PRESS.
Müller, A. C., and S. Guido. 2016. Introduction to machine learning with python: A guide for data scientists. 1st ed. Newton, MA: O’Reilly Media.
Mussone, L., A. Ferrari, and M. Oneta. 1999. “An analysis of urban collisions using an artificial intelligence model.” Accid. Anal. Prev. 31 (6): 705–718. https://doi.org/10.1016/S0001-4575(99)00031-7.
Oh, J., C. Lyon, S. Washington, B. Persaud, and J. Bard. 2003. “Validation of FHWA crash models for rural intersections: Lesions learned.” Transp. Res. Rec. 1840 (1): 41–49. https://doi.org/10.3141/1840-05.
Olutayo, V. A., and A. A. Eludire. 2014. “Traffic accident analysis using decision trees and neural networks.” Int. J. Inf. Technol. Comput. Sci. 6 (May): 22–28. https://doi.org/10.5815/ijitcs.2014.02.03.
Pedregosa, F., et al. 2011. “Scikit-learn: Machine learning in python.” J. Mach. Learn. Res. 12 (Nov): 2825–2830.
Raybaut, P. 2009. “Spyder-documentation.” Accessed December 3, 2020. https://www.spyder-ide.org.
Sacchi, E., T. Sayed, and K. El-Basyouny. 2015. “Multivariate full Bayesian hot spot identification and ranking: New technique.” Transp. Res. Rec. 2515 (1): 1–9. https://doi.org/10.3141/2515-01.
Sawalha, Z., and T. Sayed. 2006. “Traffic accident modeling: Some statistical issues.” Can. J. Civ. Eng. 33 (9): 1115–1124. https://doi.org/10.1139/l06-056.
Schölkopf, B., A. J. Smola, R. C. Williamson, and P. L. Bartlett. 2000. “New support vector algorithms.” Neural Comput. 12 (5): 1207–1245. https://doi.org/10.1162/089976600300015565.
Singh, G., S. N. Sachdeva, and M. Pal. 2018. Support vector machine model for prediction of accidents on non-urban sections of highways. Proc. Inst. Civ. Eng. Transp. 171 (5): 253–263. https://doi.org/10.1680/jtran.17.00019.
Suykens, J. A. K., T. V. Gestel, J. D. Brabanter, B. D. Moor, and J. Vanderwalle. 2002. Least squares support vector machines. Singapore: World Scientific.
Xie, Y., D. Lord, and Y. Zhang. 2007. “Predicting motor vehicle collisions using Bayesian neural network models: An empirical analysis.” Accid. Anal. Prev. 39 (5): 922–933. https://doi.org/10.1016/j.aap.2006.12.014.
Zeng, Q., and H. Huang. 2014. “A stable and optimized neural network model for crash injury severity prediction.” Accid. Anal. Prev. 73 (Dec): 351–358. https://doi.org/10.1016/j.aap.2014.09.006.
Zhang, Y., and Y. Xie. 2007. “Forecasting of short-term freeway volume with v-support vector machines.” Transp. Res. Rec. 2024 (1): 92–99. https://doi.org/10.3141/2024-11.
Information & Authors
Information
Published In
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Sep 22, 2021
Accepted: May 6, 2022
Published online: Jul 14, 2022
Published in print: Sep 1, 2022
Discussion open until: Dec 14, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Mohammadreza Koloushani, Seyed Reza Abazari, Omer Arda Vanli, Eren Erman Ozguven, Ren Moses, Rupert Giroux, Benjamin Jacobs, Determination of Optimal Spatial Sample Sizes for Fitting Negative Binomial-Based Crash Prediction Models with Consideration of Statistical Modeling Assumptions, Sustainability, 10.3390/su152014731, 15, 20, (14731), (2023).