Automatic Calibration for Monocular Cameras in Highway Scenes via Vehicle Vanishing Point Detection
Publication: Journal of Transportation Engineering, Part A: Systems
Volume 149, Issue 7
Abstract
Automatic camera calibration is a fundamental technology for 3D traffic parameter extraction. With the popularity of pan-tilt-zoom cameras, this technique demonstrates great potential to enhance traffic safety and efficiency, especially for highways. This paper aims to present a fully automatic calibration method for surveillance cameras in highway scenes. Our system is divided into two stages. In the first stage, a deep convolution neural network was used to estimate a pair of orthogonal vanishing points from multiple vehicles. This process transformed vanishing point detection into an estimation of vehicle direction, which was further determined by introducing the central residual mechanism. In the diamond space, the straight lines formed by these directions accumulated the final positions of the vanishing points. More importantly, we proposed a novel algorithm for estimating the lane width using vehicle trajectories in the second stage. It can be used to calculate the camera height, making the calibration fully automated. We also corrected the distorted lens using vehicle trajectories. Comprehensive experiments were conducted on the proposed data set and the BoxCars116k data set. The results indicate that the composite mechanism (i.e., classification and central residual) significantly improves the accuracy and robustness of the vanishing point estimation. Combined with automatic camera height estimation, our technology is superior to the most representative methods in calibration performance. Since it does not have any constraints on road geometry and camera placement, our approach applies to most highway surveillance systems.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (Grant No. 62072053).
References
Aggarwal, N., and W. C. Karl. 2000. “Line detection in images through regularized hough transform.” In Vol. 3 of Proc., 2000 Int. Conf. on Image Processing (Cat. No.00CH37101), 873–876. New York: IEEE.
Álvarez, S., D. F. Llorca, and M. A. Sotelo. 2013. “Camera auto-calibration using zooming and zebra-crossing for traffic monitoring applications.” In Proc., 16th Int. IEEE Conf. on Intelligent Transportation Systems (ITSC 2013), 608–613. New York: IEEE.
Bartl, V., J. Paňhel, P. Dobe, R. Juránek, and A. Herout. 2020. “Automatic camera calibration by landmarks on rigid objects.” Mach. Vis. Appl. 32 (1): 2. https://doi.org/10.1007/s00138-020-01125-x.
Bhardwaj, R., G. K. Tummala, G. Ramalingam, R. Ramjee, and P. Sinha. 2018. “Autocalib: Automatic traffic camera calibration at scale.” ACM Trans. Sens. Network 14 (3–4): 1–27. https://doi.org/10.1145/3199667.
Bochkovskiy, A., C. Y. Wang, and H. Liao. 2020. “Yolov4: Optimal speed and accuracy of object detection.” Preprint, Submitted April 23, 2020. http://arxiv.org/abs/2004.10934.
Chang, C.-K., J. Zhao, and L. Itti. 2018. “Deepvp: Deep learning for vanishing point detection on 1 million street view images.” In Proc., 2018 IEEE Int. Conf. on Robotics and Automation (ICRA), 4496–4503. New York: IEEE.
Comaniciu, D., and P. Meer. 2002. “Mean shift: A robust approach toward feature space analysis.” IEEE Trans. Pattern Anal. Mach. Intell. 24 (5): 603–619. https://doi.org/10.1109/34.1000236.
Coughlan, J. M., and A. L. Yuille. 1999. “Manhattan world: Compass direction from a single image by Bayesian inference.” In Vol. 2 of Proc., 7th IEEE Int. Conf. on Computer Vision, 941–947. New York: IEEE.
Dawson, D. N., and S. T. Birchfield. 2013. “An energy minimization approach to automatic traffic camera calibration.” IEEE Trans. Intell. Transp. Syst. 14 (3): 1095–1108. https://doi.org/10.1109/TITS.2013.2253553.
Dong, R., B. Li, and Q.-M. Chen. 2009. “An automatic calibration method for PTZ camera in expressway monitoring system.” In Vol. 6 of Proc., 2009 WRI World Congress on Computer Science and Information Engineering, 636–640. New York: IEEE.
Dubská, M., A. Herout, R. Juránek, and J. Sochor. 2015. “Fully automatic roadside camera calibration for traffic surveillance.” IEEE Trans. Intell. Transp. Syst. 16 (3): 1162–1171. https://doi.org/10.1109/TITS.2014.2352854.
Gioi, R. G. V., J. Jakubowicz, J. M. Morel, and G. Randall. 2012. “Lsd: A line segment detector.” Image Processing On Line 2 (4): 35–55. https://doi.org/10.5201/ipol.2012.gjmr-lsd.
Huang, G., Z. Liu, L. Van Der Maaten, and K. Q. Weinberger. 2017. “Densely connected convolutional networks.” In Proc., 2017 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2261–2269. New York: IEEE.
Jain, A. G., and N. Saunier. 2020. “Autocamera Calibration for traffic surveillance cameras with wide angle lenses.” Preprint, submitted January 20, 2020. http://arxiv.org/abs/2001.07243.
Kanhere, N. K., and S. T. Birchfield. 2010. “A taxonomy and analysis of camera calibration methods for traffic monitoring applications.” IEEE Trans. Intell. Transp. Syst. 11 (2): 441–452. https://doi.org/10.1109/TITS.2010.2045500.
Lepetit, V., F. Moreno-Noguer, and P. Fua. 2009. “Epnp: An accurate o(n) solution to the pnp problem.” Int. J. Comput. Vis. 81 (2): 155–166. https://doi.org/10.1007/s11263-008-0152-6.
Li, Y., F. Zhu, Y. Ai, and F.-Y. Wang. 2007. “On automatic and dynamic camera calibration based on traffic visual surveillance.” In Proc., 2007 IEEE Intelligent Vehicles Symp., 358–363. New York: IEEE.
Liu, W., D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, and A. C. Berg. 2016. Ssd: Single shot multibox detector. Cham, Switzerland: Springer.
Lu, L., and F. Dai. 2023. “Automated visual surveying of vehicle heights to help measure the risk of overheight collisions using deep learning and view geometry.” Comput.-Aided Civ. Infrastruct. Eng. 38 (2): 194–210. https://doi.org/10.1111/mice.12842.
Lu, X., J. Yao, K. Li, and L. Li. 2015. “Cannylines: A parameter-free line segment detector.” In Proc., 2015 IEEE Int. Conf. on Image Processing (ICIP), 507–511. New York: IEEE.
Ronda, J. I., and A. Valdés. 2019. “Geometrical analysis of polynomial lens distortion models.” J. Math. Imaging Vis. 61 (3): 252–268. https://doi.org/10.1007/s10851-018-0833-x.
Schoepflin, T., and D. Dailey. 2003. “Dynamic camera calibration of roadside traffic management cameras for vehicle speed estimation.” IEEE Trans. Intell. Transp. Syst. 4 (2): 90–98. https://doi.org/10.1109/TITS.2003.821213.
Sochor, J., R. Juranek, and A. Herout. 2017. “Traffic surveillance camera calibration by 3d model bounding box alignment for accurate vehicle speed measurement.” Comput. Vision Image Understanding 161 (Aug): 87–98. https://doi.org/10.1016/j.cviu.2017.05.015.
Sochor, J., J. Špaňhel, and A. Herout. 2019. “Boxcars: Improving fine-grained recognition of vehicles using 3-D bounding boxes in traffic surveillance.” IEEE Trans. Intell. Transp. Syst. 20 (1): 97–108. https://doi.org/10.1109/TITS.2018.2799228.
Tang, Z., Y.-S. Lin, K.-H. Lee, J.-N. Hwang, J.-H. Chuang, and Z. Fang. 2016. “Camera self-calibration from tracking of moving persons.” In Proc., 23rd Int. Conf. on Pattern Recognition (ICPR), 265–270. New York: IEEE.
Tardif, J.-P. 2009. “Non-iterative approach for fast and accurate vanishing point detection.” In Proc., 2009 IEEE 12th Int. Conf. on Computer Vision, 1250–1257. New York: IEEE.
Thi, T. H., S. Lu, and J. Zhang. 2008. “Self-calibration of traffic surveillance camera using motion tracking.” In Proc., 2008 11th Int. IEEE Conf. on Intelligent Transportation Systems, 304–309. New York: IEEE.
Wang, N., H. Du, Y. Liu, Z. Tang, and J.-N. Hwang. 2018. “Self-calibration of traffic surveillance cameras based on moving vehicle appearance and 3-D vehicle modeling.” In Proc., 25th IEEE Int. Conf. on Image Processing (ICIP), 3064–3068. New York: IEEE.
Wang, Y., X. Lu, Z. Ling, Y. Yang, Z. Zhang, and K. Wang. 2015. “A method to calibrate vehicle-mounted cameras under urban traffic scenes.” IEEE Trans. Intell. Transp. Syst. 16 (6): 3270–3279. https://doi.org/10.1109/TITS.2015.2430617.
Wojke, N., A. Bewley, and D. Paulus. 2017. “Simple online and realtime tracking with a deep association metric.” In Proc., 2017 IEEE Int. Conf. on Image Processing (ICIP), 3645–3649. New York: IEEE.
Zhai, M., S. Workman, and N. Jacobs. 2016. “Detecting vanishing points using global image context in a non-Manhattan World.” In Proc., 2016 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 5657–5665. New York: IEEE.
Zhang, Z., T. Tan, K. Huang, and Y. Wang. 2013. “Practical camera calibration from moving objects for traffic scene surveillance.” IEEE Trans. Circuits Syst. Video Technol. 23 (3): 518–533. https://doi.org/10.1109/TCSVT.2012.2210670.
Zheng, Y., and S. Peng. 2014. “A practical roadside camera calibration method based on least squares optimization.” IEEE Trans. Intell. Transp. Syst. 15 (2): 831–843. https://doi.org/10.1109/TITS.2013.2288353.
Information & Authors
Information
Published In
Journal of Transportation Engineering, Part A: Systems
Volume 149 • Issue 7 • July 2023
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Mar 5, 2022
Accepted: Nov 2, 2022
Published online: Apr 27, 2023
Published in print: Jul 1, 2023
Discussion open until: Sep 27, 2023
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Jie Shen, Lang Jiao, Cong Zhang, Keran Peng, Monocular 3D object detection for construction scene analysis, Computer-Aided Civil and Infrastructure Engineering, 10.1111/mice.13143, 39, 9, (1370-1389), (2023).