Intelligent Noncontact Structural Displacement Detection Method Based on Computer Vision and Deep Learning
Publication: Journal of Construction Engineering and Management
Volume 150, Issue 10
Abstract
Accurate identification of structural displacements is important for structural state assessment and performance evaluation. This paper proposes a real-time structural displacement detection model based on computer vision and deep learning. The model consists of three stages: identification, tracking, and displacement resolution. First, the displacement target is identified and tracked by the improved YOLO v7 algorithm and the improved DeepSORT algorithm. Then, the Euclidean distance method based on inverse perspective mapping (IPM-ED) is proposed for the analytical conversion of the displacement. Next, the accuracy and effectiveness of this displacement detection model are evaluated through four groups of bamboo axial compression tests. A comparative analysis is conducted between the IPM-ED displacement analysis method and the commonly used ED displacement analysis method. Finally, the robustness of this method is tested by using a cable breakage test of a cable dome structure as an application case. The research results demonstrate that the maximum average error of the four groups of bamboo displacement tests is only 3.10 mm, and the maximum relative error of peak displacement is only 6.54 mm. The RMSE basically stays around 3.5 mm. The maximum displacement error in the application case is only 4.91 mm, with a maximum MAPE of 4.94%. In addition, the error percentage under the IPM-ED algorithm is basically within 5%, while the error percentage of the ED algorithm is more than 10%. The method in this paper achieves efficient and intelligent identification of structural displacements in a non-contact manner. The proposed method is suitable for environments where the contact displacement sensor is easily affected by vibration, the site is complex and requires additional displacement sensor fixing equipment, the displacement sensor with super-high structure is unsafe to deploy, and the contact displacement sensor in narrow space is inconvenient to deploy, so it has broad application prospects.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This research was supported by Hebei Province Full-time Top-level Talents Introduction Project, No. 2020HBQZYC013 and the China Construction Sixth Engineering Bureau Research and Development Project, No. CSCEC6B-2022-Z-2. Thanks to Elsevier language editing services.
References
Bai, X. B., N. Yang, and Q. S. Yang. 2018. “Temperature effect on the structural strains of an ancient Tibetan building based on long-term monitoring data.” Earthquake Eng. Eng. Vibr. 17 (3): 641–657. https://doi.org/10.1007/s11803-018-0437-x.
Chen, I. H., S. C. Ho, and M. B. Su. 2020. “Computer vision application programming for settlement monitoring in a drainage tunnel.” Autom. Constr. 110 (Apr): 1–9. https://doi.org/10.1016/j.autcon.2019.103011.
Chen, Z., F. Zhang, H. Liu, L. Wang, Q. Zhang, and L. Guo. 2023. “Real-time detection algorithm of helmet and reflective vest based on improved YOLOv5.” J. Real-Time Image Process. 20 (1): 4. https://doi.org/10.1007/s11554-023-01268-w.
Dai, L., N. Yang, L. Zhang, Q. Yang, and S. S. Law. 2016. “Monitoring crowd load effect on typical ancient Tibetan building.” Struct. Control Health Monit. 23 (7): 998–1014. https://doi.org/10.1002/stc.1821.
Dong, C. Z., and F. N. Catbas. 2019. “A non-target structural displacement measurement method using advanced feature matching strategy.” Adv. Struct. Eng. 22 (16): 3461–3472. https://doi.org/10.1177/1369433219856171.
Feng, D., and M. Q. Feng. 2018. “Computer vision for SHM of civil infrastructure: From dynamic response measurement to damage detection: A review.” Eng. Struct. 156 (Apr): 105–117. https://doi.org/10.1016/j.engstruct.2017.11.018.
Feng, D. M., and M. Q. Feng. 2017. “Experimental validation of cost-effective vision-based structural health monitoring.” Mech. Syst. Signal Process. 88 (May): 199–211. https://doi.org/10.1016/j.ymssp.2016.11.021.
Kim, H., S. H. Sim, and F. S. Billie. 2022. “Automated concrete crack evaluation using stereo vision with two different focal lengths.” Autom. Constr. 135 (Aug): 104136. https://doi.org/10.1016/j.autcon.2022.104136.
Lin, X., C. T. Li, V. Sanchez, and C. Maple. 2021. “On the detection-to-track association for online multi-object tracking.” Pattern Recognit. Lett. 146 (Jun): 200–207. https://doi.org/10.1016/j.patrec.2021.03.022.
Liu, Y., Y. Deng, and C. S. Cai. 2015. “Deflection monitoring and assessment for a suspension bridge using a connected pipe system: A case study in China.” Struct. Control Health Monit. 22 (12): 1408–1425. https://doi.org/10.1002/stc.1751.
Martinez, D., A. Malekjafarian, and E. Obrien. 2020. “Bridge health monitoring using deflection measurements under random traffic.” Struct. Control Health Monit. 27 (9): e2593. https://doi.org/10.1002/stc.2593.
Qiu, Q. W., and D. Lau. 2023. “Real-time detection of cracks in tiled sidewalks using YOLO-based method applied to unmanned aerial vehicle (UAV) images.” Autom. Constr. 147 (Sep): 104745. https://doi.org/10.1016/j.autcon.2023.104745.
Tan, Y., R. Cai, J. Li, P. Chen, and M. Wang. 2021. “Automatic detection of sewer defects based on improved you only look once algorithm.” Autom. Constr. 131 (Nov): 103912. https://doi.org/10.1016/j.autcon.2021.103912.
Wang, L., H. Liu, Z. Chen, F. Zhang, and L. Guo. 2023. “Combined digital twin and hierarchical deep learning approach for intelligent damage identification in cable dome structure.” Eng. Struct. 274 (Jan): 115172. https://doi.org/10.1016/j.engstruct.2022.115172.
Xu, H., W. X. Ren, and Z. C. Wang. 2015. “Deflection estimation of bending beam structures using fiber bragg grating strain sensors.” Adv. Struct. Eng. 18 (3): 395–403. https://doi.org/10.1260/1369-4332.18.3.395.
Xu, Y., J. Brownjohn, and D. Kong. 2018. “A non-contact vision-based system for multipoint displacement monitoring in a cable-stayed footbridge.” Struct. Control Health Monit. 25 (5): 1–23. https://doi.org/10.1002/stc.2155.
Yang, J., H. Ge, J. Yang, Y. Tong, and S. Su. 2022. “Online multi-object tracking using multi-function integration and tracking simulation training.” Appl. Intell. 52 (2): 1268–1288. https://doi.org/10.1007/s10489-021-02457-5.
Yang, N. A., D. Wang, T. Li, and F. Bai. 2023. “Deformation monitoring of ancient buildings based on computer vision method.” [In Chinese.] J. Build. Struct. 44 (1): 192–202. https://doi.org/10.14006/j.jzjgxb.2021.0499.
Yang, Y. B., B. Q. Wang, Z. L. Wang, K. Shi, H. Xu, B. Zhang, and Y. T. Wu. 2020. “Bridge surface roughness identified from the displacement influence lines of the contact points by two connected vehicles.” Int. J. Struct. Stab. Dyn. 20 (14): 2043003. https://doi.org/10.1142/S0219455420430038.
Ye, X. W., C. Z. Dong, and T. Liu. 2016. “Image-based structural dynamic displacement measurement using different multi-object tracking algorithms.” Smart Struct. Syst. 17 (6): 935–956. https://doi.org/10.12989/sss.2016.17.6.935.
Ye, X. W., Y. Q. Ni, T. T. Wai, K. Y. Wong, X. M. Zhang, and F. Xu. 2013. “A Vision-based system for dynamic displacement measurement of long-span bridges: Algorithm and verification.” Smart Struct. Syst. 12 (3/4): 363–379. https://doi.org/10.12989/sss.2013.12.3_4.363.
Yi, T. H., H. N. Li, and M. Gu. 2013a. “Experimental assessment of high-rate GPS receivers for deformation monitoring of bridge.” Measurement 46 (1): 420–432. https://doi.org/10.1016/j.measurement.2012.07.018.
Yi, T. H., H. N. Li, and M. Gu. 2013b. “Recent research and applications of GPS-based monitoring technology for high-rise structures.” Struct. Control Health Monit. 20 (5): 649–670. https://doi.org/10.1002/stc.1501.
Zhang, X., T. Wan, Z. Wu, and B. Du. 2022. “Real-time detector design for small targets based on bi-channel feature fusion mechanism.” Appl. Intell. 52 (3): 2775–2784. https://doi.org/10.1007/s10489-021-02545-6.
Zhou, Y., L. Zhang, T. Liu, and S. Gong. 2018. “Structural system identification based on computer vision.” [In Chinese.] China Civ. Eng. J. 51 (11): 17–23. https://doi.org/10.15951/j.tmgcxb.20180716.006.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 150 • Issue 10 • October 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Dec 12, 2023
Accepted: May 7, 2024
Published online: Jul 24, 2024
Published in print: Oct 1, 2024
Discussion open until: Dec 24, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.