Noncontact Dynamic Three-Component Displacement Measurement with a Dual Stereovision–Enabled Uncrewed Aerial System
Publication: Journal of Engineering Mechanics
Volume 150, Issue 8
Abstract
Measuring the dynamic displacements of a structure provides a comprehensive understanding of the structure, especially when subjected to different types of dynamic loading (i.e., wind, traffic, impact loads, blast loads, etc.). Despite their usefulness, direct displacement measurements are typically not collected due to the cumbersome logistical issues of sensor placement and maintenance and the impracticality of instrumenting contact-based sensors across all significant structures. In this context, this study proposes a novel dual stereovision technique to measure the dynamic displacement of structures using a portable, noncontact measurement system that involves an uncrewed aerial system (UAS) and four optical cameras. One pair of cameras tracks the three-component (, , and ) motion of a region of interest (ROI) on a structure with respect to the UAS system, and the other pair of cameras measure the six degrees of freedom motion (6-DOF) (both rotational and translational motion) of the UAS system by tracking a stationary reference. The motion of the UAS is then compensated for to recover the true dynamic displacement of the ROI. The proposed dual stereovision technique realizes simultaneous measurement of all three components of displacements of the structure and 6-DOF of UAS motion through a mathematically elegant process. The unique dual stereovision technique allows flexibility in choosing a global reference coordinate system, greatly enhancing the feasibility of applying the new technology in various field environments. This new technique has overcome the major challenge of significant UAS motions in full-scale applications. Furthermore, this technique relies on natural features and eliminates the requirement of artificial targets on the structure, permitting applications to difficult-to-access structures.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
The camera data used in the stationary point measurement experiment are available from the corresponding author upon reasonable request.
Acknowledgments
The work presented in this paper was conducted with support from Colorado State University (CSU) and the Mountain-Plains Consortium, a University Transportation Center funded by the US Department of Transportation (FASTACT Grant No. 69A3551747108). The contents of this paper reflect the views of the authors, who are responsible for the facts and accuracy of the information presented. Additionally, the authors would like to acknowledge the Colorado State University’s Drone Center and Mr. Christopher Robertson for providing the UAVs and expertise, as well as Mr. Todd Atadero of Colorado State University for his insight and assistance on the experiments in this study.
References
Adamczyk, M., P. Liberadzki, and R. Sitnik. 2018. “Temperature compensation method for digital cameras in 2D and 3D measurement applications.” Sensors 18 (11): 3685. https://doi.org/10.3390/s18113685.
Aghlara, R., and M. M. Tahir. 2018. “Measurement of strain on concrete using an ordinary digital camera.” Meas. J. Int. Meas. Confederation 126 (10): 398–404. https://doi.org/10.1016/j.measurement.2018.05.066.
Anwar, S., S. Khan, and N. Barnes. 2020. “A deep journey into super-resolution: A survey.” ACM Comput. Surv. 53 (3): 1–34.
Arun, K. S., T. S. Huang, and S. D. Blostein. 1987. “Least-squares fitting of two 3-D point sets.” IEEE Trans. Pattern Anal. Mach. Intell. 9 (5): 698–700. https://doi.org/10.1109/TPAMI.1987.4767965.
Bouguet, J.-Y. 2001. “Pyramidal implementation of the Affine Lucas Kanade feature tracker description of the algorithm.” Intel Corp. 5 (1–10): 4.
Brincker, R., L. Zhang, and P. Andersen. 2000. “Modal identification from ambient responses using frequency domain decomposition.” In Proc., Int. Modal Analysis Conf. San Antonio: Aalborg Universitet.
Brown, R., K. Gaebler, C. Shield, and L. Linderman. 2019. Displacement monitoring of I-35W Saint Anthony falls bridge with current vibration-based system. Minneapolis: Minnesota DOT.
Catt, S., B. Fick, M. Hoskins, J. Praski, and J. Baquersad. 2019. “Development of a semi-autonomous drone for structural health monitoring of structures using digital image correlation (DIC).” In Proc., 36th IMAC, A Conf. and Exposition on Structural Dynamics 2018. Berlin: Springer.
Celik, O., C.-Z. Dong, and F. N. Catbas. 2020. “Computer vision-based human comfort assessment of stadiums.” J. Perform. Constr. Facil. 34 (2): 04020005. https://doi.org/10.1061/(ASCE)CF.1943-5509.0001345.
Dong, C., S. Bas, and F. N. Catbas. 2023. “Applications of computer vision-based structural monitoring on long-span bridges in Turkey.” Sensors 23 (19): 8161. https://doi.org/10.3390/s23198161.
Dong, C., X. Ye, and T. Jin. 2018. “Identification of structural dynamic characteristics based on machine vision technology.” Measurement 126 (10): 405–416. https://doi.org/10.1016/j.measurement.2017.09.043.
Garg, P., R. Nasimi, A. Ozdagli, S. Zhang, D. D. L. Mascarenas, M. Reda Taha, and F. Moreu. 2020. “Measuring transverse displacements using unmanned aerial systems laser doppler vibrometer (UAS-LDV): Development and field validation.” Sensors 20 (21): 6051. https://doi.org/10.3390/s20216051.
Guo, Y., and A. Kareem. 2015. “System identification through nonstationary response: Wavelet and transformed singular value decomposition–based approach.” J. Eng. Mech. 141 (7): 04015013. https://doi.org/10.1061/(ASCE)EM.1943-7889.0000905.
Guo, Y., and A. Kareem. 2016. “Non-stationary frequency domain system identification using time-frequency representations.” Mech. Syst. Signal Process. 72–73 (5): 712–726. https://doi.org/10.1016/j.ymssp.2015.10.031.
Guo, Y., A. Kareem, Y. Ni, and W. Liao. 2012. “Performance evaluation of Canton Tower under winds based on full-scale data.” J. Wind Eng. Ind. Aerodyn. 104–106 (Jun): 116–128. https://doi.org/10.1016/j.jweia.2012.04.001.
Guo, Y., D. K. Kwon, and A. Kareem. 2016. “Near-real-time hybrid system identification framework for civil structures with application to Burj Khalifa.” J. Struct. Eng. 142 (2): 04015132. https://doi.org/10.1061/(ASCE)ST.1943-541X.0001402.
Harris, C., and M. Stephens. 1988. “A combined corner and edge detector.” In Proc., Alvey Vision Conf. Manchester, UK: Alvety Vision Club.
Havaran, A., and M. Mahmoudi. 2020. “Extracting structural dynamic properties utilizing close photogrammetry method.” Measurement 150 (1): 107092. https://doi.org/10.1016/j.measurement.2019.107092.
Horn, B. K., and B. G. Schunck. 1981. “Determining optical flow.” Artif. Intell. 17 (1–3): 185–203. https://doi.org/10.1016/0004-3702(81)90024-2.
Hoskere, V., J. Park, H. Yoon, and B. Spencer. 2019. “Vision-based modal survey of civil infrastructure using unmanned aerial vehicles.” J. Struct. Eng. 145 (7): 04019062. https://doi.org/10.1061/(ASCE)ST.1943-541X.0002321.
Kalaitzakis, M., S. Kattil, N. Vitzilaios, D. Rizos, and M. Sutton. 2019. “Dynamic structural health monitoring using a DIC-enabled drone.” In Proc., 2019 Int. Conf. on Unmanned Aircraft Systems. New York: IEEE.
Kijewski-Correa, T., A. Kareem, and M. Kochly. 2006. “Experimental verification and full-scale deployment of global positioning systems to monitor the dynamic response of tall buildings.” J. Struct. Eng. 132 (8): 1242–1253. https://doi.org/10.1061/(ASCE)0733-9445(2006)132:8(1242).
Kijewski-Correa, T., D. Kwon, A. Kareem, A. Bentz, Y. Guo, S. Bobby, and A. Abdelrazaq. 2013. “Smartsync: An integrated real-time structural health monitoring and structural identification system for tall buildings.” J. Struct. Eng. 139 (10): 1675–1687. https://doi.org/10.1061/(ASCE)ST.1943-541X.0000560.
Kromanis, R., and C. Forbes. 2019. “A low-cost robotic camera system for accurate collection of structural response.” Inventions 4 (3): 47. https://doi.org/10.3390/inventions4030047.
Kromanis, R., and P. Kripakaran. 2017. “Data-driven approaches for measurement interpretation: Analysing integrated thermal and vehicular response in bridge structural health monitoring.” Adv. Eng. Inf. 34 (10): 46–59. https://doi.org/10.1016/j.aei.2017.09.002.
Kromanis, R., and P. Kripakaran. 2021. “A multiple camera position approach for accurate displacement measurement using computer vision.” J. Civ. Struct. Health Monit. 11 (3): 661–678. https://doi.org/10.1007/s13349-021-00473-0.
Li, R., B. Zeng, and M. Liou. 1994. “A new three-step search algorithm for block motion estimation.” IEEE Trans. Circuits Syst. Video Technol. 4 (4): 438–442. https://doi.org/10.1109/76.313138.
Li, Z., and C. Chang. 2012. “Tracking of structural dynamic characteristics using recursive stochastic subspace identification and instrumental variable technique.” J. Eng. Mech. 138 (6): 591–600. https://doi.org/10.1061/(ASCE)EM.1943-7889.0000370.
Liu, X., S. J. Dyke, A. Lenjani, I. Bilionis, X. Zhang, and J. Choi. 2023. “Automated image localization to support rapid building reconnaissance in a large-scale area.” Comput.-Aided Civ. Infrastruct. Eng. 38 (1): 3–25. https://doi.org/10.1111/mice.12828.
Liu, X., X. Tong, W. Lu, S. Liu, B. Huang, P. Tang, and T. Guo. 2020. “High-speed videogrammetric measurement of the deformation of shaking table multi-layer structures.” Measurement 154 (15): 107486. https://doi.org/10.1016/j.measurement.2020.107486.
Mondal, T. G., and M. R. Jahanshahi. 2022. “Chapter 12—Applications of depth sensing for advanced structural condition assessment in smart cities.” In The rise of smart cities, 305–318. New York: Butterworth-Heinemann.
Moschas, F., and S. Stiros. 2011. “Measurement of the dynamic displacements and of the modal frequencies of a short-span pedestrian bridge using GPS and an accelerometer.” Eng. Struct. 33 (1): 10–17. https://doi.org/10.1016/j.engstruct.2010.09.013.
Mousa, M., M. Yussof, U. Udi, F. Nazri, M. Kamarudin, G. Parke, L. Assi, and S. Ghahari. 2021. “Application of digital image correlation in structural health monitoring of bridge infrastructures: A review.” Infrastructures 6 (12): 176. https://doi.org/10.3390/infrastructures6120176.
Nasimi, R., F. Moreu, and G. Fricke. 2023. “Sensor equipped UAS for non-contact bridge inspections: Field application.” Sensors 23 (1): 470. https://doi.org/10.3390/s23010470.
Perry, B., and Y. Guo. 2021. “A portable three-component displacement measurement technique using an unmanned aerial vehicle (UAV) and computer vision: A proof of concept.” Measurement 176 (5): 109222. https://doi.org/10.1016/j.measurement.2021.109222.
Rainieri, C., and G. Fabbrocino. 2010. “Automated output-only dynamic identification of civil engineering structures.” Mech. Syst. Signal Process. 24 (3): 678–695. https://doi.org/10.1016/j.ymssp.2009.10.003.
Ribeiro, D., R. Santos, R. Cabral, G. Saramago, P. Montenegro, H. Carvalho, J. Correia, and R. Calçada. 2021. “Non-contact structural displacement measurement using unmanned aerial vehicles and video-based systems.” Mech. Syst. Signal Process. 160 (11): 107869. https://doi.org/10.1016/j.ymssp.2021.107869.
Roberts, G., X. Meng, and A. Dodson. 2004. “Integrating a global positioning system and accelerometers to monitor the deflection of bridges.” J. Surv. Eng. 130 (2): 65–72. https://doi.org/10.1061/(ASCE)0733-9453(2004)130:2(65).
Sabharwal, C., and Y. Guo. 2019. “Tracking the 6-DOF flight trajectory of windborne debris using stereophotogrammetry.” Infrastructures 4 (4): 66. https://doi.org/10.3390/infrastructures4040066.
Tjahjadi, M. E., and F. Handoko. 2017. “Precise wide baseline stereo image matching for compact digital cameras.” In Proc., 2017 4th Int. Conf. on Electrical Engineering, Computer Science and Informatics (EECSI), 1–6. New York: IEEE.
Wu, J. 2019. “Optimal continuous unit quaternions from rotation matrices.” J. Guid. Control Dyn. 42 (4): 919–922. https://doi.org/10.2514/1.G004043.
Yoon, H., V. Hoskere, J. Park, and B. Spencer. 2017. “Cross-correlation-based structural system identification using unmanned aerial vehicles.” Sensors 17 (9): 2075. https://doi.org/10.3390/s17092075.
Yoon, H., J. Shin, and B. Spencer. 2018. “Structural displacement measurement using an unmanned aerial system.” Comput.-Aided Civ. Infrastruct. Eng. 33 (3): 183–192. https://doi.org/10.1111/mice.12338.
Zhengyou Z. 1999. “Flexible camera calibration by viewing a plane from unknown orientations.” In Proc., 7th IEEE Int. Conf. on Computer Vision. New York: IEEE.
Information & Authors
Information
Published In
Journal of Engineering Mechanics
Volume 150 • Issue 8 • August 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Jul 17, 2023
Accepted: Feb 18, 2024
Published online: Jun 7, 2024
Published in print: Aug 1, 2024
Discussion open until: Nov 7, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.