Abstract

The current paper implements a methodology for automatically detecting vehicle maneuvers from vehicle telemetry data under naturalistic driving settings. Previous approaches have treated vehicle maneuver detection as a classification problem, although both time series segmentation and classification are required since input telemetry data are continuous. Our objective is to develop an end-to-end pipeline for the frame-by-frame annotation of naturalistic driving studies videos into various driving events including stop and lane-keeping events, lane changes, left-right turning movements, and horizontal curve maneuvers. To address the time series segmentation problem, the study developed an energy-maximization algorithm (EMA) capable of extracting driving events of varying durations and frequencies from continuous signal data. To reduce overfitting and false alarm rates, heuristic algorithms were used to classify events with highly variable patterns such as stops and lane-keeping. To classify segmented driving events, four machine-learning models were implemented, and their accuracy and transferability were assessed over multiple data sources. The duration of events extracted by EMA was comparable to actual events, with accuracies ranging from 59.30% (left lane change) to 85.60% (lane-keeping). Additionally, the overall accuracy of the 1D-convolutional neural network model was 98.99%, followed by the long-short-term-memory model at 97.75%, then the random forest model at 97.71%, and the support vector machine model at 97.65%. These model accuracies were consistent across different data sources. The study concludes that implementing a segmentation-classification pipeline significantly improves both the accuracy of driver maneuver detection and the transferability of shallow and deep ML models across diverse datasets.

Get full access to this article

View all available purchase options and get full access to this article.

Data Availability Statement

Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.

References

Aboah, A. 2022. “NDS automatic video annotation.” Accessed January 1, 2022. https://youtu.be/JAuCfRGnLBI.
Aboah, A., M. Boeding, and Y. Adu-Gyamfi. 2021. “Mobile sensing for multipurpose applications in transportation.” J. Big Data Anal. Transp. 9 (Nov): 1–3. https://doi.org/10.48550/arXiv.2106.10733.
Antin, J. F., S. Lee, M. A. Perez, T. A. Dingus, J. M. Hankey, and A. Brach. 2019. “Second strategic highway research program naturalistic driving study methods.” Saf. Sci. 119 (Nov): 2–10. https://doi.org/10.1016/j.ssci.2019.01.016.
ASIRT. 2021. “ASIRT—Association for safe international road travel.” Accessed November 16, 2021. https://www.asirt.org/.
Atia, A., M.-S. Mostafa, A. Hamdy Ali, and M. Sami. 2017. “Recognizing driving behavior and road anomaly using smartphone sensors driving events recognition using smartphone sensors.” Int. J. Ambient Comput. Intell. 8 (3): 22–37. https://doi.org/10.4018/IJACI.2017070102.
Ayres, G., B. Wilson, and J. LeBlanc. 2004. “Method for identifying vehicle movements for analysis of field operational test data.” Transp. Res. Rec. 1886 (1): 92–100. https://doi.org/10.3141/1886-12.
Bakhit, P., O. Osman, and S. Ishak. 2017. “Detecting imminent lane change maneuvers in connected vehicle environments.” Transp. Res. Rec. 2645 (1): 168–175. https://doi.org/10.3141/2645-18.
Barnard, Y., F. Utesch, N. van Nes, R. Eenink, and M. Baumann. 2016. “The study design of UDRIVE: The naturalistic driving study across Europe for cars, trucks and scooters.” Eur. Transport Res. Rev. 8 (2): 14. https://doi.org/10.1007/s12544-016-0202-z.
Bejani, M. M., and M. Ghatee. 2019. “Convolutional neural network with adaptive regularization to classify driving styles on smartphones.” IEEE Trans. Intell. Transp. Syst. 21 (2): 543–552. https://doi.org/10.1109/TITS.2019.2896672.
Benmimoun, M., F. Fahrenkrog, A. Zlocki, and L. Eckstein. 2011. “Incident detection based on vehicle CAN-data within the large-scale field operational test euroFOT.” In Proc., 22nd Enhanced Safety of Vehicles Conf. (ESV 2011). Washington, DC: National Highway Traffic Safety Administration.
Bhoraskar, R., N. Vankadhara, B. Raman, and P. Kulkarni. 2012. “Wolverine: Traffic and road condition estimation using smartphone sensors.” In Proc., 2012 Fourth Int. Conf. on Communication Systems and Networks (COMSNETS 2012), 1–6. New York: IEEE.
Bogard, S. 1999. Analysis of data on speed-change and lane-change behavior in manual and ACC driving. Newbury Park, CA: SAGE.
Breiman, L. 2001. “Random forests.” Mach. Learn. 45 (1): 5–32. https://doi.org/10.1023/A:1010933404324.
Carvalho, E., B. V. Ferreira, J. Ferreira, C. De Souza, H. V. Carvalho, Y. Suhara, A. S. Pentland, and G. Pessin. 2017. “Exploiting the use of recurrent neural networks for driver behavior profiling.” In Proc., 2017 Int. Joint Conf. on Neural Networks (IJCNN), 3016–3021. New York: IEEE.
Charlton, J. L., et al. 2019. “Changes in driving patterns of older Australians: Findings from the Candrive/Ozcandrive cohort study.” Saf. Sci. 119 (Nov): 219–226. https://doi.org/10.1016/j.ssci.2018.11.008.
Chen, D., K. T. Cho, S. Han, Z. Jin, and K. G. Shin. 2015. “Invisible sensing of vehicle steering with smartphones.” In Proc., 13th Annual Int. Conf. on Mobile Systems, Applications, and Services, 1–13. New York: Association for Computing Machinery. https://doi.org/10.1145/2742647.2742659.
Cortes, C., and V. Vapnik. 1995. “Support-vector networks.” Mach. Learn. 20 (3): 273–297. https://doi.org/10.1007/BF00994018.
Dingus, T. A., et al. 2006. The 100-car naturalistic driving study phase II-Results of the 100-car field experiment. Washington, DC: DOT, National Highway Traffic Safety Administration.
Drincic, A., M. Rizzo, C. Desouza, and J. Merickel. 2020. “Digital health technologies, diabetes, and driving (meet your new backseat driver).” In Diabetes digital health, 219–230. Amsterdam, Netherlands: Elsevier.
Freund, Y., and R. E. Schapire. 1996. “Experiments with a new boosting algorithm.” Citeseer 96 (Jul): 148–156.
Fridman, L., et al. 2019. “MIT advanced vehicle technology study: Large-scale naturalistic driving study of driver behavior and interaction with automation.” IEEE Access 7 (Jul): 102021–102038. https://doi.org/10.1109/ACCESS.2019.2926040.
Gao, D., W. Li, J. Duan, and B. Zheng. 2009. “A practical method of road detection for intelligent vehicle.” In Proc., 2009 IEEE Int. Conf. on Automation and Logistics, ICAL 2009, 980–985. New York: IEEE. https://doi.org/10.1109/ICAL.2009.5262562.
Global Status Report on Road Safety. 2021. “Global status report on road safety—Google Search.” Accessed November 16, 2021. https://www.google.com/search?q=global+status+report+on+road+safety.
Hankey, J., M. Perez, and J. McClafferty. 2016. Description of the SHRP 2 naturalistic database and the crash, near-crash, and baseline data sets. Blacksburg, VA: Virginia Tech Transportation Institute.
Houenou, A., P. Bonnifait, V. Cherfaoui, and W. Yao. 2013. “Vehicle trajectory prediction based on motion model and maneuver recognition.” In Proc., IEEE Int. Conf. on Intelligent Robots and Systems, 4363–4369. New York: IEEE.
Júnior, J. F., E. Carvalho, B. V. Ferreira, C. de Souza, Y. Suhara, A. Pentland, and G. Pessin. 2017. “Driver behavior profiling: An investigation with different smartphone sensors and machine learning.” PLoS One 12 (4): e0174959. https://doi.org/10.1371/journal.pone.0174959.
Kang, L., and S. Banerjee. 2017. “Practical driving analytics with smartphone sensors.” In Proc., 2017 IEEE Vehicular Networking Conf. (VNC), 303–310. New York: IEEE.
Karatas, C., L. Liu, H. Li, J. Liu, Y. Wang, S. Tan, J. Yang, Y. Chen, M. Grusteser, and R. Martin. 2016. “Leveraging wearables for steering and driver tracking.” In Proc., IEEE INFOCOM 2016-The 35th Annual IEEE Int. Conf. on Computer Communications, 1–9. New York: IEEE.
Kumar, P., M. Perrollaz, S. Lefevre, and C. Laugier. 2013. “Learning-based approach for online lane change intention prediction.” In Proc., 2013 IEEE Intelligent Vehicles Symp. (IV), 797–802. New York: IEEE.
Larue, G. S., et al. 2018. “Australian naturalistic driving study (ANDS): Using 20,000 trips to get a glimpse at locations and speeds where data was collected.” In Proc., 2018 Australasian Road Safety Conf. Newbury Park, CA: SAGE.
Lerner, N., J. Jenness, J. Singer, S. Klauer, S. Lee, M. Donath, M. Manser, and N. Ward. 2010. An exploration of vehicle-based monitoring of novice teen drivers: Final report. Washington, DC: National Highway Traffic Safety Administration.
Li, P., M. Abdel-Aty, and Z. Islam. 2021. “Driving maneuvers detection using semi-supervised long short-term memory and smartphone sensors.” Transp. Res. Rec.: J. Transp. Res. Board 2675 (9): 1386–1397. https://doi.org/10.1177/03611981211007483.
Li, W., X. Gong, Y. Wang, and P. Liu. 2014. “A lane marking detection and tracking algorithm based on sub-regions.” In Proc., ICCSS 2014—Int. Conf. on Informative and Cybernetics for Computational Social Systems, 68–73. Piscataway, NJ: Institute of Electrical and Electronics Engineers Inc.
Mandalia, H. M., and D. D. Salvucci. 2005. “Using support vector machines for lane-change detection.” In Proc., Human Factors and Ergonomics Society, 1965–1969. Thousand Oaks, CA: SAGE.
McGehee, D. V., M. Raby, C. Carney, J. D. Lee, and M. L. Reyes. 2007. “Extending parental mentoring using an event-triggered video intervention in rural teen drivers.” J. Saf. Res. 38 (2): 215–227. https://doi.org/10.1016/j.jsr.2007.02.009.
Miller, R. J., and G. Srinivasan. 2005. “Determination of lane change maneuvers using naturalistic driving data.” In Proc., 19th Int. Technical Conf. on the Enhanced Safety of Vehicles (ESV), Washington, DC. Newbury Park, CA: SAGE.
Mohamed, A., A. Issam, B. Mohamed, and B. Abdellatif. 2015. “Real-time detection of vehicles using the haar-like features and artificial neuron networks.” Procedia Comput. Sci. 73 (Jan): 24–31. https://doi.org/10.1016/j.procs.2015.12.044.
Morris, B., A. Doshi, and M. Trivedi. 2011. “Lane change intent prediction for driver assistance: On-road design and evaluation.” In Proc., IEEE Intelligent Vehicles Symp., 895–901. New York: IEEE.
Naiel, M. A., M. O. Ahmad, and M. N. S. Swamy. 2014. “Vehicle detection using TD2DHOG features.” In Proc., 2014 IEEE 12th Int. New Circuits and Systems Conf., NEWCAS 2014, 389–392. New York: IEEE.
NHTSA (National Highway Traffic Safety Administration), U. 1999. Evaluation of the intelligent cruise control system: Volume 1: Study results. Washington, DC: NHTSA.
Ohn-Bar, E., A. Tawari, S. Martin, and M. M. Trivedi. 2014. “Predicting driver maneuvers by learning holistic features.” In Proc., IEEE Intelligent Vehicles Symp., 719–724. New York: IEEE.
Olson, R., R. Hanowski, J. Hickman, and J. Bocanegra. 2009. Driver distraction in commercial vehicle operations. Washington, DC: Dept. of Transportation, Federal Motor Carrier Safety Administration.
Panichpapiboon, S., and P. Leakkaw. 2020. “Lane change detection with smartphones: A steering wheel-based approach.” IEEE Access 8 (May): 91076–91088. https://doi.org/10.1109/ACCESS.2020.2994352.
Perslev, M., M. H. Jensen, S. Darkner, P. J. Jennum, and C. Igel. 2019. “U-time: A fully convolutional network for time series segmentation applied to sleep staging.r.” Preprint, submitted October 24, 2019. https://arxiv.org/abs/1910.11162.
Pilgerstorfer, M., K. Runda, C. Brandstätter, and M. Christoph. 2012. “Deliverable 6.3, report on small scale naturalistic driving pilot.” In Vol. 6 of Small scale naturalistic driving pilot, deliverable. Loughborough, UK: Loughborough Univ.
Rotaru, C., T. Graf, and J. Zhang. 2008. “Color image segmentation in HSI space for automotive applications.” J. Real-Time Image Process. 3 (4): 311–322. https://doi.org/10.1007/s11554-008-0078-9.
Saiprasert, C., T. Pholprasit, and W. Pattara-Atikom. 2013. “Detecting driving events using smartphone.” In Vol. 11 of Proc., 20th ITS World Congress, 11. Tokyo: Japan Organizing Committee.
Saiprasert, C., T. Pholprasit, and S. Thajchayapong. 2017. “Detection of driving events using sensory data on smartphone.” Int. J. Intell. Transp. Syst. Res. 15 (1): 17–28. https://doi.org/10.1007/s13177-015-0116-5.
Son, J., H. Yoo, S. Kim, and K. Sohn. 2015. “Real-time illumination invariant lane detection for lane departure warning system.” Expert Syst. Appl. 42 (4): 1816–1824. https://doi.org/10.1016/j.eswa.2014.10.024.
Sun, R., Q. Cheng, F. Xie, W. Zhang, T. Lin, and W. Y. Ochieng. 2019. “Combining machine learning and dynamic time wrapping for vehicle driving event detection using smartphones.” IEEE Trans. Intell. Transp. Syst. 22 (1): 194–207. https://doi.org/10.1109/TITS.2019.2955760.
Tavakoli, A., and A. Heydarian. 2021. “Multimodal driver state modeling through unsupervised learning.” Accid. Anal. Prev. 170 (Jun): 106640. https://doi.org/10.1016/j.aap.2022.106640.
Toledo, T., and D. Zohar. 2007. “Modeling duration of lane changes.” Transp. Res. Rec. 1999 (1): 71–78. https://doi.org/10.3141/1999-08.
Xu, H., X. Wang, H. Huang, K. Wu, and Q. Fang. 2009. “A fast and stable lane detection method based on B-spline curve.” In Proc., 2009 IEEE 10th Int. Conf. on Computer-Aided Industrial Design and Conceptual Design: E-Business, Creative Design, Manufacturing—CAID and CD’2009, 1036–1040. New York: IEEE.
Xuan, Y., and B. Coifman. 2006. “Lane change maneuver detection from probe vehicle DGPS data.” In Proc., IEEE Conf. on Intelligent Transportation Systems, ITSC, 624–629. New York: IEEE.
Yang, D., X. Qiu, Y. Liu, C. Wen, L. Zhu, and X. Hong. 2017. Modeling the discretionary lane-changing decision behavior on freeways using random forest theory. Newbury Park, CA: SAGE.
You, C. W., et al. 2013. “Carsafe app: Alerting drowsy and distracted drivers using dual cameras on smartphones.” In Proc., 11th Annual Int. Conf. on Mobile Systems, Applications, and Services, 13–26. New York: Association for Computing Machinery. https://doi.org/10.1145/2462456.2465428.
Yu, J., Z. Chen, Y. Zhu, Y. Chen, L. Kong, and M. Li. 2016. “Fine-grained abnormal driving behaviors detection and identification with smartphones.” IEEE Trans. Mob. Comput. 16 (8): 2198–2212. https://doi.org/10.1109/TMC.2016.2618873.
Zheng, J., K. Suzuki, and M. Fujita. 2014. “Predicting driver’s lane-changing decisions using a neural network model.” Simul. Modell. Pract. Theory 42 (Mar): 73–83. https://doi.org/10.1016/j.simpat.2013.12.007.

Information & Authors

Information

Published In

Go to Journal of Transportation Engineering, Part A: Systems
Journal of Transportation Engineering, Part A: Systems
Volume 149Issue 3March 2023

History

Received: Jan 5, 2022
Accepted: Nov 7, 2022
Published online: Dec 29, 2022
Published in print: Mar 1, 2023
Discussion open until: May 29, 2023

Permissions

Request permissions for this article.

Authors

Affiliations

Ph.D. Student, Dept. of Civil and Environmental Engineering, Univ. of Missouri-Columbia, E25O9 Lafferre Hall, Columbia, MO 65211 (corresponding author). ORCID: https://orcid.org/0000-0002-1605-1545. Email: [email protected]
Assistant Professor, Dept. of Civil and Environmental Engineering, Univ. of Missouri-Columbia, E25O9 Lafferre Hall, Columbia, MO 65211. ORCID: https://orcid.org/0000-0002-1924-9792. Email: [email protected]
Professor, Electrical Engineering and Computer Science, Syracuse Univ., Lincoln, NE 68588 ORCID: https://orcid.org/0000-0002-1430-1555. Email: [email protected]
Assistant Professor, Dept. of Neurological Sciences, Univ. of Nebraska Medical Center, Omaha, NE 68198-8440. ORCID: https://orcid.org/0000-0002-0394-9110. Email: [email protected]
Matt Rizzo
Professor, Dept. of Neurological Sciences, Univ. of Nebraska Medical Center, Omaha, NE 68198-8440.
Anuj Sharma [email protected]
Professor, Dept. of Civil and Environmental Engineering, Iowa State Univ, Ames, IA 50010. Email: [email protected]

Metrics & Citations

Metrics

Citations

Download citation

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited by

  • Driver Maneuver Interaction Identification with Anomaly-Aware Federated Learning on Heterogeneous Feature Representations, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 10.1145/3631421, 7, 4, (1-28), (2024).
  • Classification of Human Driver Distraction Using 3D Convolutional Neural Networks, Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 10.1177/21695067231192576, 67, 1, (2088-2094), (2023).
  • DeepSegmenter: Temporal Action Localization for Detecting Anomalies in Untrimmed Naturalistic Driving Videos, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 10.1109/CVPRW59228.2023.00565, (5359-5365), (2023).
  • Vehicle Warning System Based on Road Curvature Effect Using CNN and LSTM Neural Networks, 18th International Conference on Soft Computing Models in Industrial and Environmental Applications (SOCO 2023), 10.1007/978-3-031-42536-3_25, (256-266), (2023).

View Options

Get Access

Access content

Please select your options to get access

Log in/Register Log in via your institution (Shibboleth)
ASCE Members: Please log in to see member pricing

Purchase

Save for later Information on ASCE Library Cards
ASCE Library Cards let you download journal articles, proceedings papers, and available book chapters across the entire ASCE Library platform. ASCE Library Cards remain active for 24 months or until all downloads are used. Note: This content will be debited as one download at time of checkout.

Terms of Use: ASCE Library Cards are for individual, personal use only. Reselling, republishing, or forwarding the materials to libraries or reading rooms is prohibited.
ASCE Library Card (5 downloads)
$105.00
Add to cart
ASCE Library Card (20 downloads)
$280.00
Add to cart
Buy Single Article
$35.00
Add to cart

Get Access

Access content

Please select your options to get access

Log in/Register Log in via your institution (Shibboleth)
ASCE Members: Please log in to see member pricing

Purchase

Save for later Information on ASCE Library Cards
ASCE Library Cards let you download journal articles, proceedings papers, and available book chapters across the entire ASCE Library platform. ASCE Library Cards remain active for 24 months or until all downloads are used. Note: This content will be debited as one download at time of checkout.

Terms of Use: ASCE Library Cards are for individual, personal use only. Reselling, republishing, or forwarding the materials to libraries or reading rooms is prohibited.
ASCE Library Card (5 downloads)
$105.00
Add to cart
ASCE Library Card (20 downloads)
$280.00
Add to cart
Buy Single Article
$35.00
Add to cart

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share