Proposing a Comprehensive Evaluation Method for AI-Based Traffic Detection System and Post-Processing Method Using Physical Constraints

Automatic collection of accurate vehicle traffic data, such as vehicle speed, volume, and classification, is critical for the function of smart cities. With the wide availability of traffic cameras, efforts have been made to use AI object detection algorithms for this task. However, the common performance measures used by AI/computer vision communities are mostly concerned with frame level detection performance, which does not directly translate to the accuracy of the traffic data extracted. Therefore, to understand and improve the performance of different AI implementations for traffic data extraction, there is a need for a consistent method that comprehensively evaluates the system performance under diverse conditions. This study seeks to propose a comprehensive evaluation method for AI-based traffic information extraction system. A case study was completed by assessing the performance of a previously developed traffic data extraction system. To demonstrate the benefit of comprehensive evaluation, a headway post-processing method was proposed based on the evaluation results and applied to the system. The case study results showed that the AI-based system produced more accurate results in both non-saturated and saturated traffic flow, tangent roadway geometries, daytime lighting, and lower camera angles, while producing less accurate results in nighttime conditions and at higher camera angles. The headway post-processing method effectively reduced vehicle volume overcounting error to within a 3.8% error regardless of condition. The outcome of this study would support the development of standardized evaluation methodology and public data set to consistently compare the performance of the traffic data extraction system with different AI implementations and drive the advancement in this field.