A Benchmark Data Set for Vision-Based Traffic Load Monitoring in a Cable-Stayed Bridge
Publication: Journal of Bridge Engineering
Volume 29, Issue 2
Abstract
Traffic load monitoring based on deep learning and computer vision has garnered significant attention in bridge engineering worldwide. Unlike traditional traffic load monitoring systems, computer vision-based techniques can accurately extract the spatiotemporal load distribution across the entire bridge in an autonomous manner. However, many of the related studies in the literature used data sets that were collected from a few specific areas of different bridges, and there are very limited data sets that provide complete coverage of the entire bridge, making a detailed comparison of different computer vision methods difficult. This paper presents a benchmark data set that provides a series of annotations and field measurements required for traffic load detection, tracking, and continuous monitoring on the bridge. The data set was collected by five cameras and two weigh-in-motion systems installed on a cable-stayed bridge and is divided into three subsets. The first subset contains over 32,000 images and annotation files of 11 types of vehicle-related targets, which are necessary for the training of vehicle detection models. The second subset consists of photos of the calibration board and coordinates of reference points that are used for camera calibration. The last subset is designated for the field verification of various algorithms, providing synchronized vehicle weight data and monitoring videos covering the whole bridge. To the author’s knowledge, this data set is the first open-source data set for vision-based traffic load monitoring in a bridge, which will have tremendous value in promoting research in the area of innovative bridge health monitoring technologies. Details of this data set will be available in the public domain through a Zenodo data repository.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
The data collection in this study was authorized by the local municipal administration in China. To protect the privacy of individuals and entities included in this data set, all the identifiable and privacy information has been anonymized using mosaic processing. All the data used during the study are available in a repository online in accordance with funder data retention policies. The data are available at Zenedo by the following link: https://zenodo.org/record/7924653.
Acknowledgments
This study is supported by the Project of the Ministry of Transport “Research on key technologies of intelligent highways with large traffic volume” (Project number: 2020-ZD3-025); China Railway Engineering Design and Consulting Group Co., Ltd. technology development project “Research and development of bridge management and maintenance system based on perceptual neural network”; the Fundamental Research Funds for the Central Universities (20210205); the National Natural Science Foundation of China (Grant number: 51878490); and the National Key R&D Program of China (2017YFF0205605).
References
Barker, M. G., and J. A. Puckett. 2019. “Unique truck loads on interstates in Wyoming and similar states and the impact on bridge safety design.” J. Bridge Eng. 24 (2): 04018110. https://doi.org/10.1061/(ASCE)BE.1943-5592.0001312.
Chen, Z., H. Li, Y. Bao, N. Li, and Y. Jin. 2016. “Identification of spatio-temporal distribution of vehicle loads on long-span bridges using computer vision technology.” Struct. Control Health Monit. 23 (3): 517–534. https://doi.org/10.1002/stc.1780.
Cordts, M., M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. 2016. “The cityscapes dataset for semantic urban scene understanding.” In Proc., of the Institute of Electrical and Electronics Engineers Conf. on Computer Vision and Pattern Recognition, 3213–3223. Piscataway, NJ: Institute of Electrical and Electronics Engineers (IEEE).
Dan, D., L. Ge, and X. Yan. 2019. “Identification of moving loads based on the information fusion of weigh-in-motion system and multiple camera machine vision.” Measurement 144: 155–166. https://doi.org/10.1016/j.measurement.2019.05.042.
Dan, D., Y. Ying, and L. Ge. 2022. “Digital twin system of bridges group based on machine vision fusion monitoring of bridge traffic load.” IEEE Trans. Intell. Transp. Syst. 23 (11): 22190–22205. https://doi.org/10.1109/TITS.2021.3130025.
Deng, J., W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009. “ImageNet: A large-scale hierarchical image database.” In Proc., Institute of Electrical and Electronics Engineers Conf. on Computer Vision and Pattern Recognition, 248–255. Piscataway, NJ: Institute of Electrical and Electronics Engineers (IEEE).
Deng, L., L. Nie, W. Zhong, and W. Wang. 2021. “Developing fatigue vehicle models for bridge fatigue assessment under different traffic conditions.” J. Bridge Eng. 26 (2): 04020122. https://doi.org/10.1061/(ASCE)BE.1943-5592.0001675.
Ge, L., D. Dan, and H. Li. 2020. “An accurate and robust monitoring method of full-bridge traffic load distribution based on YOLO-v3 machine vision.” Struct. Control Health Monit. 27 (12): e2636. https://doi.org/10.1002/stc.2636.
Ge, L., D. Dan, Z. Liu, and X. Ruan. 2022. “Intelligent simulation method of bridge traffic flow load combining machine vision and weigh-in-motion monitoring.” IEEE Trans. Intell. Transp. Syst. 23 (9): 15313–15328. https://doi.org/10.1109/TITS.2022.3140276.
Ge, L., K. Y. Koo, M. Wang, J. Brownjohn, and D. Dan. 2023. “Bridge damage detection using precise vision-based displacement influence lines and weigh-in-motion devices: Experimental validation.” Eng. Struct. 288: 116185. https://doi.org/10.1016/j.engstruct.2023.116185.
Guerrero-Gómez-Olmedo, R., B. Torre-Jiménez, R. López-Sastre, S. Maldonado-Bascón, and D. Onoro-Rubio. 2015. “Extremely overlapping vehicle counting.” In Proc., 7th Iberian Conf. on Pattern Recognition and Image Analysis, 423–431. Berlin, Germany: Springer International Publishing.
Haugen, T., J. R. Levy, E. Aakre, and M. E. P. Tello. 2016. “Weigh-in-motion equipment—Experiences and challenges.” Transp. Res. Procedia 14: 1423–1432. https://doi.org/10.1016/j.trpro.2016.05.215.
Jian, X., Y. Xia, S. Sun, and L. Sun. 2022. “Integrating bridge influence surface and computer vision for bridge weigh-in-motion in complicated traffic scenarios.” Struct. Control Health Monit. 29 (11): e3066. https://doi.org/10.1002/stc.3066.
Lin, T.-Y., M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. Lawrence Zitnick, and P. Dollár. 2014. “Microsoft coco: Common objects in context.” In Proc., 13th European Conf. Computer Vision, 740–755. Berlin, Germany: Springer International Publishing.
Luo, L., Y. Xia, A. Wang, X. Lei, X. Jian, and L. Sun. 2022. “Finite element model updating method for continuous girder bridges using monitoring responses and traffic videos.” Struct. Control Health Monit. 29 (11): e3062.
Lydon, D., M. Lydon, R. Kromanis, C.-Z. Dong, N. Catbas, and S. Taylor. 2021. “Bridge damage detection approach using a roving camera technique.” Sensors 21 (4): 1246. https://doi.org/10.3390/s21041246.
Martini, A., E. M. Tronci, M. Q. Feng, and R. Y. Leung. 2022. “A computer vision-based method for bridge model updating using displacement influence lines.” Eng. Struct. 259: 114129. https://doi.org/10.1016/j.engstruct.2022.114129.
Micu, E. A., A. Malekjafarian, E. J. OBrien, M. Quilligan, R. McKinstray, E. Angus, M. Lydon, and F. N. Catbas. 2019. “Evaluation of the extreme traffic load effects on the Forth Road Bridge using image analysis of traffic data.” Adv. Eng. Software 137: 102711. https://doi.org/10.1016/j.advengsoft.2019.102711.
OBrien, E. J., and B. Enright. 2013. “Using weigh-in-motion data to determine aggressiveness of traffic for bridge loading.” J. Bridge Eng. 18 (3): 232–239. https://doi.org/10.1061/(ASCE)BE.1943-5592.0000368.
Pan, Y., D. Wang, Y. Dong, and B. Peng. 2022. “A novel vision-based framework for identifying dynamic vehicle loads on long-span bridges: A case study of Jiangyin Bridge, China.” IEEE Trans. Intell. Transp. Syst. 23 (8): 10441–10457. https://doi.org/10.1109/tits.2021.3094504.
Ricolfe-Viala, C., and A.-J. Sanchez-Salmeron. 2010. “Lens distortion models evaluation.” Appl. Opt. 49 (30): 5914–5928. https://doi.org/10.1364/AO.49.005914.
Sadhu, A., J. E. Peplinski, A. Mohammadkhorasani, and F. Moreu. 2023. “A review of data management and visualization techniques for structural health monitoring using BIM and virtual or augmented reality.” J. Struct. Eng. 149 (1): 03122006. https://doi.org/10.1061/(ASCE)ST.1943-541X.0003498.
Siavashi, S., and C. D. Eamon. 2019. “Development of traffic live-load models for bridge superstructure rating with RBDO and best selection approach.” J. Bridge Eng. 24 (8): 04019084. https://doi.org/10.1061/(ASCE)BE.1943-5592.0001457.
Singh, P., S. Mittal, and A. Sadhu. 2023. “Recent advancements and future trends in indirect bridge health monitoring.” Pract. Period. Struct. Des. Constr. 28 (1): 03122008. https://doi.org/10.1061/PPSCFX.SCENG-1259.
Sony, S., S. Laventure, and A. Sadhu. 2019. “A literature review of next-generation smart sensing technology in structural health monitoring.” Struct. Control Health Monit. 26 (3): e2321. https://doi.org/10.1002/stc.2321.
Sun, L., Z. Shang, Y. Xia, S. Bhowmick, and S. Nagarajaiah. 2020. “Review of bridge structural health monitoring aided by big data and artificial intelligence: From condition assessment to damage detection.” J. Struct. Eng. 146 (5): 04020073. https://doi.org/10.1061/(ASCE)ST.1943-541X.0002535.
Sun, Z., and X.-W. Ye. 2022. “Incorporating site-specific weigh-in-motion data into fatigue life assessment of expansion joints under dynamic vehicle load.” Eng. Struct. 255: 113941. https://doi.org/10.1016/j.engstruct.2022.113941.
Wang, F.-Y., and Y.-L. Xu. 2019. “Traffic load simulation for long-span suspension bridges.” J. Bridge Eng. 24 (5): 05019005. https://doi.org/10.1061/(ASCE)BE.1943-5592.0001381.
Yang, D.-H., Z.-X. Guan, T.-H. Yi, H.-N. Li, and Y.-S. Ni. 2022. “Fatigue evaluation of bridges based on strain influence line loaded by elaborate stochastic traffic flow.” J. Bridge Eng. 27 (9): 04022082. https://doi.org/10.1061/(ASCE)BE.1943-5592.0001929.
Zaurin, R., T. Khuc, and F. N. Catbas. 2016. “Hybrid sensor-camera monitoring for damage detection: Case study of a real bridge.” J. Bridge Eng. 21 (6): 05016002. https://doi.org/10.1061/(asce)be.1943-5592.0000811.
Zhang, Z. 2000. “A flexible new technique for camera calibration.” IEEE Trans. Pattern Anal. Mach. Intell. 22 (11): 1330–1334. https://doi.org/10.1109/34.888718.
Zhou, Y., Y. Pei, Z. Li, L. Fang, Y. Zhao, and W. Yi. 2020. “Vehicle weight identification system for spatiotemporal load distribution on bridges based on non-contact machine vision technology and deep learning algorithms.” Measurement 159: 107801. https://doi.org/10.1016/j.measurement.2020.107801.
Zhu, J., X. Li, C. Zhang, and T. Shi. 2021. “An accurate approach for obtaining spatiotemporal information of vehicle loads on bridges based on 3D bounding box reconstruction with computer vision.” Measurement 181: 109657. https://doi.org/10.1016/j.measurement.2021.109657.
Information & Authors
Information
Published In
Journal of Bridge Engineering
Volume 29 • Issue 2 • February 2024
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Feb 21, 2023
Accepted: Aug 10, 2023
Published online: Nov 20, 2023
Published in print: Feb 1, 2024
Discussion open until: Apr 20, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.