Modeling Human Learning and Cognition Structure: Application to Driver Behavior in Dilemma Zone
Publication: Journal of Transportation Engineering
Volume 142, Issue 11
Abstract
In transportation studies, modeling human learning and decision-making processes plays a key role in developing realistic safety countermeasures and appropriate crash-mitigation strategies. In this study, a human learning model was created that captures the cognitive structure of human memory. The relationship between long-term and short-term memories was incorporated into a reinforcement learning technique to construct the human learning model. The model was then applied to dilemma zone data collected in a simulator study. Dilemma zone is an area of roadway ahead of the signalized intersection in which drivers have difficulty deciding whether to stop or proceed through at the onset of yellow. Driver choice behavior and learning process in dilemma zones was modeled, taking into account drivers’ experiences at the previous intersections, and was compared to a pure machine learning model. The results of the model revealed lower and faster-merging errors when human learning was considered in training agents. The human learning model for dilemma zones presented here could be used to evaluate dilemma zone mitigation algorithms by considering their effects on driver agents.
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgments
This study was supported in part by the Mid-Atlantic Universities Transportation Center as well as the Virginia Center for Transportation Innovation and Research (VCTIR). The authors are solely responsible for the material in this paper and the views are not necessarily those of the supporting agencies.
References
Abbas, M., and Ghanipoor Machiani, S. (2013). “Modeling the dynamics of driver’s dilemma zone perception using agent based modeling techniques.” Int. J. Transp. Spec. Issue Agent Based Model. Transp. Plann. Oper., in press.
Abbas, M., Ghanipoor Machiani, S., Garvey, P. M., Farkas, A., and Lord-Attivor, R. (2014). “Modeling the dynamics of driver’s dilemma zone perception using machine learning methods for safer intersection control.” U.S. Dept. of Transportation, Research & Innovative Technology, Washington, DC.
Abbas, M., Linsen, C., Higgs, B., Medina, A., and Yang, C. Y. D. (2011). “Agent-based evaluation of driver heterogeneous behavior during safety-critical events.” 2011 14th Int. IEEE Conf. on Intelligent Transportation Systems (ITSC), IEEE, Piscataway, NJ.
Abdoos, M., Mozayani, N., and Bazzan, A. L. C. (2014). “Hierarchical control of traffic signals using -learning with tile coding.” Appl. Intell., 40(2), 201–213.
Adam, Z., Abbas, M., and Li, P. (2009). “Evaluating green-extension policies with reinforcement learning and Markovian traffic state estimation.” Transp. Res. Rec., 2128(-1), 217–225.
Ahn, H.-i. (2010). “Modeling and analysis of affective influences on human experience, prediction, decision making, and behavior.” Massachusetts Institute of Technology, Cambridge, MA.
Allos, A. E., and Al-Hadithi, M. I. (1992). “Driver behaviour during onset of amber at signalised junctions.” Traffic Eng. Control, 33(5), 312–317.
AnyLogic. (2016). “Multimethod simulation software.” 〈http://www.anylogic.com/〉 (May 15, 2016).
Barto, A. G., and Şimşek, Ö. (2005). “Intrinsic motivation for reinforcement learning systems.” Proc., 13th Yale Workshop on Adaptive and Learning Systems, Yale Univ., New Haven, CT, 113–118.
Camerer, C., and Hua Ho, T. (1999). “Experience-weighted attraction learning in normal form games.” Econometrica, 67(4), 827–874.
Castro, R., Kalish, C., Nowak, R., Qian, R., Rogers, T., and Zhu, X. (2008). “Human active learning.” Advances in Neural Information Processing Systems (NIPS) 2008, Vancouver, Canada.
Chin, Y. K., Bolong, N., Kiring, A., Yang, S. S., and Teo, K. T. K. (2011). “-learning based traffic optimization in management of signal timing plan.” Int. J. Simul. Syst. Sci. Technol., 12(3), 29–35.
Chin, Y. K., Wei Yeang, K., Wei Leong, K., Min Keng, T., and Teo, K. T. K. (2012). “-learning traffic signal optimization within multiple intersections traffic network.” 2012 6th UKSim/AMSS European Symp. on Computer Modeling and Simulation (EMS), IEEE, Piscataway, NJ.
Chong, L., Abbas, M., Higgs, B., Medina, A., and Yang, C. Y. D. (2011a). “A revised reinforcement learning algorithm to model complicated vehicle continuous actions in traffic.” 2011 14th Int. IEEE Conf. on Intelligent Transportation Systems (ITSC), IEEE, Piscataway, NJ.
Chong, L., Abbas, M., Higgs, B., Medina, A., and Yang, C. Y. D. (2011b). “Determination and optimization of reinforcement learning parameters for driver actions in traffic.” 2011 14th Int. IEEE Conf. on Intelligent Transportation Systems (ITSC), IEEE, Piscataway, NJ.
Elhenawy, M., Jahangiri, A., Rakha, H. A., and El-Shawarby, I. (2015). “Modeling driver stop/run behavior at the onset of a yellow indication considering driver run tendency and roadway surface conditions.” Accid. Anal. Prev., 83, 90–100.
Elmitiny, N., Yan, X., Radwan, E., Russo, C., and Nashar, D. (2010). “Classification analysis of driver’s stop/go decision and red-light running violation.” Accid. Anal. Prev., 42(1), 101–111.
Erev, I., and Roth, A. E. (1998). “Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria.” Am. Econ. Rev., 88, 4, 848–881.
Gates, T. J., Noyce, D. A., Laracuente, L., and Nordheim, E. V. (2007). “Analysis of driver behavior in dilemma zones at signalized intersections.” Transp. Res. Rec., 2030, 29–39.
Ghanipoor Machiani, S., and Abbas, M. (2014a). “Dynamic driver’s perception of dilemma zone: Experimental design and analysis of driver’s learning in a simulator study.” 93nd Annual Meeting of the Transportation Research Board, Transportation Research Board, Washington, DC.
Ghanipoor Machiani, S., and Abbas, M. (2014b). “Predicting drivers decision in dilemma zone in a driving simulator environment using canonical discriminant analysis.” 93nd Annual Meeting of the Transportation Research Board, Transportation Research Board, Washington, DC.
Ghanipoor Machiani, S., and Abbas, M. (2015b). “Assessment of driver stopping prediction models before and after the onset of yellow using two driving simulator datasets.” Accid. Anal. Prev., in press.
Ghanipoor Machiani, S., and Abbas, M. (2015a). “Safety surrogate histograms (SSH): A novel real-time safety assessment of dilemma zone related conflicts at signalized intersections.” Accid. Anal. Prev., in press.
Hertwig, R., Barron, G., Weber, E. U., and Erev, I. (2004). “Decisions from experience and the effect of rare events in risky choice.” Psychol. Sci., 15(8), 534–539.
Huitt, W., and Hummel, J. (2006). “An overview of the behavioral perspective.” 〈http://www.edpsycinteractive.org/topics/behavior/behovr.html〉 (May 15, 2016).
Hurwitz, D. S., Wang, H., Knodler, M. A., Jr., Ni, D., and Moore, D. (2012). “Fuzzy sets to describe driver behavior in the dilemma zone of high-speed signalized intersections.” Transp. Res. Part F: Traffic Psychol. Behav., 15(2), 132–143.
Jahangiri, A., Rakha, H., and Dingus, T. A. (2015). “Predicting red-light running violations at signalized intersections using machine learning techniques.” Transportation Research Board 93rd Annual Meeting, Transportation Research Board, Washington, DC.
Kahneman, D., and Tversky, A. (1979). “Prospect theory: An analysis of decision under risk.” Econometrica, 47(2), 263–291.
Kikuchi, S., Perincherry, V., Chakroborty, P., and Takahashi, H. (1993). “Modeling of driver anxiety during signal change intervals.” Transp. Res. Rec., 1399, 27–35.
Killeen, P. R. (1994). “Mathematical principles of reinforcement.” Behav. Brain Sci., 17(1), 105–135.
Lee, S., and Son, Y.-J. (2008). “Integrated human decision making model under belief-desire-intention framework for crowd simulation.” Simulation Conf., 2008 Winter Simulation Conf., IEEE, Piscataway, NJ, 886–894.
Lee, S., and Son, Y.-J. (2009). “Dynamic learning in human decision behavior for evacuation scenarios under BDI framework.” Proc., 2009 INFORMS Simulation Society Research Workshop, INFORMS Simulation Society, Catonsville, MD, 96–100.
Lehman, J. F., Laird, J., and Rosenbloom, P. (2006). “A gentle introduction to soar, an architecture for human cognition: 2006 update.” 〈http://web.eecs.umich.edu/~soar/sitemaker/docs/misc/GentleIntroduction-2006.pdf〉.
Li, P., and Abbas, M. (2010). “Stochastic dilemma hazard model at high-speed signalized intersections.” J. Transp. Eng., 448–456.
Newell, A. (1990). Unified theories of cognition, Harvard University Press, Cambridge, MA.
Olson, P. L., and Rothery, R. W. (1961). “Driver response to the amber phase of traffic signals.” Oper. Res., 9(5), 650–663.
Parsonson, P. S., Roseveare, R. W., and Thomas, Jr., J. R. (1974). “Small-area detection at intersection approaches.” J. Transp. Eng., 44(5), 8–17.
Roth, A. E., and Erev, I. (1995). “Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term.” Games Econ. Behav., 8(1), 164–212.
Singh, S., Barto, A. G., and Chentanez, N. (2004). “Intrinsically motivated reinforcement learning.” Advances in Neural Information Processing Systems 17: Proc., 2004 Conf., MIT Press, Cambridge, MA.
Singh, S., Lewis, R. L., Barto, A. G., and Sorg, J. (2010). “Intrinsically motivated reinforcement learning: An evolutionary perspective.” IEEE Trans. Auton. Mental Dev., 2(2), 70–82.
Stout, A., Konidaris, G. D., and Barto, A. G. (2005). “Intrinsically motivated reinforcement learning: A promising framework for developmental robot learning.” Proc., AAAI Spring Symp. on Developmental Robotics, Stanford Univ., Stanford, CA.
Sutton, R. S., and Barto, A. G. (1998). Introduction to reinforcement learning, MIT Press.
Watkins, C. J. C. H. (1989). “Learning from delayed rewards.” Ph.D. thesis, King’s College, Cambridge Univ., U.K.
Wavetronix. (2016). 〈http://www.wavetronix.com/en〉 (May 15, 2016).
Zegeer, C., and Deen, R. (1978). “Green-extension systems at high-speed intersections.” ITE J., 48(11), 19–24.
Zegeer, C. V. (1977). “Effectiveness of green-extension systems at highspeed intersections.” Kentucky Dept. of Transportation, Frankfort, KY, 31.
Zhao, X., and Son, Y. (2008). “BDI-based human decision-making model in automated manufacturing systems.” Int. J. Model. Simul., 28(3), 347–356.
Information & Authors
Information
Published In
Journal of Transportation Engineering
Volume 142 • Issue 11 • November 2016
Copyright
© 2016 American Society of Civil Engineers.
History
Received: Oct 7, 2015
Accepted: Apr 20, 2016
Published online: Jul 1, 2016
Published in print: Nov 1, 2016
Discussion open until: Dec 1, 2016
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.