Vision-Based Modal Survey of Civil Infrastructure Using Unmanned Aerial Vehicles
Publication: Journal of Structural Engineering
Volume 145, Issue 7
Abstract
Computer vision techniques for extracting dynamic structural displacements from videos are gaining increasing acceptance for the purposes of system identification and structural health monitoring. However, the application of video-based techniques for modal analysis of full-scale civil infrastructure has been limited, because obtaining measurements of all points on a large structure with a single video frame with sufficient resolution is seldom feasible. In this study, a new approach is presented to facilitate the extraction of frequencies and mode shapes of full-scale civil infrastructure from video obtained by an unmanned aerial vehicle (UAV). This approach addresses directly a number of difficulties associated with modal analysis of full-scale infrastructure using vision-based methods. The proposed approach is evaluated using a story-story shear-building model excited on a shaking table in a laboratory environment, and on a full-scale pedestrian suspension bridge. The results demonstrate the efficacy of the proposed approach.
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgments
The authors extend their sincere thanks to Amir Ibrahim for piloting the UAV flight as the remote-pilot-in-command during the field test, to Professor Mani Golparvar-Fard for providing access to a DJI Phantom 4 Pro and to members of the Smart Structures Technology Lab at the University of Illinois, Urbana-Champaign, for helping conduct the field test including Yuguang Fu, Tu Hoang, Yasutaka Narazaki, Fernando Gomez, Xinxia Li, Siang Zhou, and Professor Zhu Li.
References
ASCE. 2017. “ASCE’s 2017 infrastructure report card: Structurally deficient bridges.” Accessed October 11, 2017. https://www.infrastructurereportcard.org/cat-item/bridges/.
Cha, Y.-J., J. G. Chen, and O. Büyüköztürk. 2017. “Output-only computer vision based damage detection using phase-based optical flow and unscented Kalman filters.” Eng. Struct. 132 (Feb): 300–313. https://doi.org/10.1016/j.engstruct.2016.11.038.
Chang, C. C., and X. H. Xiao. 2010. “An integrated visual-inertial technique for structural displacement and velocity measurement.” Smart Struct. Syst. 6 (9): 1025–1039. https://doi.org/10.12989/sss.2010.6.9.1025.
Chen, J. G., A. Davis, N. Wadhwa, F. Durand, W. T. Freeman, and O. Büyüköztürk. 2017. “Video camera-based vibration measurement for civil infrastructure applications.” J. Infrastruct. Syst. 23 (3): B4016013. https://doi.org/10.1061/(ASCE)IS.1943-555X.0000348.
Chen, J. G., N. Wadhwa, Y. J. Cha, F. Durand, W. T. Freeman, and O. Buyukozturk. 2015. “Modal identification of simple structures with high-speed video using motion magnification.” J. Sound Vib. 345 (Jun): 58–71. https://doi.org/10.1016/j.jsv.2015.01.024.
Cho, S., R. K. Giles, and B. F. Spencer. 2015. “System identification of a historic swing truss bridge using a wireless sensor network employing orientation correction.” Struct. Control Health Monit. 22 (2): 255–272. https://doi.org/10.1002/stc.1672.
DJI (Dà-Jiāng Innovations Science and Technology Limited). 2018. “DJI—The future of possible.” Accessed February 12, 2018. https://www.dji.com/.
Farrar, C. R., and G. H. James III. 1997. “System identification from ambient vibration measurements on a bridge.” J. Sound Vib. 205 (1): 1–18. https://doi.org/10.1006/jsvi.1997.0977.
Feng, D., and M. Q. Feng. 2016. “Vision-based multipoint displacement measurement for structural health monitoring.” Struct. Control Health Monit. 23 (5): 876–890. https://doi.org/10.1002/stc.1819.
Fukuda, Y., M. Q. Feng, Y. Narita, S. Kaneko, and T. Tanaka. 2013. “Vision-based displacement sensor for monitoring dynamic response using robust object search algorithm.” IEEE Sensors J. 13 (12): 4725–4732. https://doi.org/10.1109/JSEN.2013.2273309.
Garrido-Jurado, S., R. Muñoz-Salinas, F. J. Madrid-Cuevas, and M. J. Marín-Jiménez. 2014. “Automatic generation and detection of highly reliable fiducial markers under occlusion.” Pattern Recognit. 47 (6): 2280–2292. https://doi.org/10.1016/j.patcog.2014.01.005.
Harris, C., and M. Stephens. 1988. “A combined corner and edge detector.” In Proc., the Alvey Vision Conference 1988, 147–151. Romsey, UK: Plessey Research Roke Manor.
Hoskere, V., Y. Narazaki, T. A. Hoang, and B. F. Spencer. 2017. “Vision-based structural inspection using multiscale deep convolutional neural networks.” In Proc., 3rd Huixian Int. Forum on Earthquake Engineering for Young Researchers. Urbana, IL: Univ. of Illinois at Urbana-Champaign.
Hoskere, V., Y. Narazaki, T. A. Hoang, and B. F. Spencer Jr. 2018. “Towards automated post-earthquake inspections with deep learning-based condition-aware models.” In Proc., 7th World Conf. on Structural Control and Monitoring, 7WCSCM. Ithaca, NY: Cornell Univ.
James III, G. H., T. G. Carne, P. J. Lauffer, and J. P. Lauffer. 1993. “The natural excitation technique (NExT) for modal parameter extraction from operating wind turbines.” Int. J. Anal. Exp. Modal Anal. 10 (4): 260–277.
Jang, S., et al. 2010. “Structural health monitoring of a cable-stayed bridge using smart sensor technology: Deployment and evaluation.” Smart Struct. Syst. 6 (5–6): 439–459. https://doi.org/10.12989/sss.2010.6.5_6.439.
Javh, J., J. Slavič, and M. Boltežar. 2017. “The subpixel resolution of optical-flow-based modal analysis.” Mech. Syst. Sig. Process. 88 (May): 89–99. https://doi.org/10.1016/j.ymssp.2016.11.009.
Jin, G. J. G., M. K. Sain, and B. F. J. Spencer. 2005. “Frequency domain system identification for controlled civil engineering structures.” IEEE Trans. Control Syst. Technol. 13 (6): 1055–1062. https://doi.org/10.1109/TCST.2005.854341.
Lewis, J. 1995. “Fast normalized cross-correlation.” Vision Interface 10 (1): 120–123.
Lynch, J. P., A. Sundararajan, K. H. Law, A. S. Kiremidjian, E. Carryer, H. Sohn, and C. R. Farrar. 2003. “Field validation of a wireless structural health monitoring system on the Alamosa Canyon Bridge.” In Vol. 5057 of Smart structures and materials: Smart systems and nondestructive evaluation for civil infrastructures, 267–278. Bellingham, WA: The International Society for Optics and Photonics.
Marquardt, D. W. 1963. “An algorithm for least-squares estimation of nonlinear parameters.” J. Soc. Ind. Appl. Math. 11 (2): 431–441. https://doi.org/10.1137/0111030.
Narazaki, Y., V. Hoskere, T. A. Hoang Jr., and B. F. Spencer. 2018a. “Automated bridge component recognition using video data.” In Proc., 7th World Conf. on Structural Control and Monitoring, 7WCSCM. Harbin, China: Harbin Institute of Technology.
Narazaki, Y., V. Hoskere, and B. F. Spencer. 2018b. “Free vibration-based system identification using temporal cross-correlations.” Struct. Control Health Monit. 25 (8): e2207. https://doi.org/10.1002/stc.2207.
Nocedal, J., and S. J. Wright. 2006. Numerical optimization. New York: Springer.
Nogueira, F. M. A., F. S. Barbosa, and L. P. S. Barra. 2005. “Evaluation of structural natural frequencies using image processing.” In Proc., EVACES 2005—Experimental Vibration Analysis For Civil Engineering Structures, 1–7. Zurich, Switzerland: International Association of Bridge and Structural Engineering.
OpenCV. 2015. “Detection of ArUco Markers.” 3.2.0. Accessed February 03, 2017. https://docs.opencv.org/3.2.0/d5/dae/tutorial_aruco_detection.html.
Rice, J. A., K. Mechitov, S.-H. Sim, T. Nagayama, S. Jang, R. Kim, B. F. Spencer, G. Agha, and Y. Fujino. 2010. “Flexible smart sensor framework for autonomous structural health monitoring.” Smart Struct. Syst. 6 (5–6): 423–438. https://doi.org/10.12989/sss.2010.6.5_6.423.
Schumacher, T., and A. Shariati. 2013. “Monitoring of structures and mechanical systems using virtual visual sensors for video analysis: Fundamental concept and proof of feasibility.” Sensors 13 (12): 16551–16564. https://doi.org/10.3390/s131216551.
Sim, S.-H., B. F. Spencer Jr., M. Zhang, and H. Xie. 2010. “Automated decentralized modal analysis using smart sensors.” Struct. Control Health Monit. 17 (8): 872–894. https://doi.org/10.1002/stc.348.
Spencer, B. F., Jr., F. Moreu, and R. E. Kim 2015. Campaign monitoring of railroad bridges in high-speed rail shared corridors using wireless smart sensors. Champaign, IL: Univ. of Illinois at Urbana-Champaign.
Spencer, B. F., Jr., V. Hoskere, and Y. Narazaki. 2019. “Advances in computer vision–based civil infrastructure inspection and monitoring.” Engineering 5 (2): 199–222. https://doi.org/10.1016/j.eng.2018.11.030.
Tomasi, C., and T. Kanade 1991. “Detection and tracking of point features. Pittsburgh: Carnegie Mellon Univ.
Wadhwa, N., M. Rubinstein, F. Durand, and W. T. Freeman. 2013. “Phase-based video motion processing.” ACM Trans. Graphics 32 (4): 1. https://doi.org/10.1145/2461912.2461966.
Yamaguchi, K. 2015. “mexopencv.” Accessed December 19, 2017. https://kyamagu.github.io/mexopencv/.
Yang, Y., C. Dorn, T. Mancini, Z. Talken, G. Kenyon, C. Farrar, and D. Mascarenas. 2017. “Blind identification of full-field vibration modes from video measurements with phase-based video motion magnification.” Mech. Syst. Sig. Process. 85: 567–590. https://doi.org/10.1016/j.ymssp.2016.08.041.
Yoon, H., H. Elanwar, H. Choi, M. Golparvar-Fard, and B. F. Spencer. 2016. “Target-free approach for vision-based structural system identification using consumer-grade cameras.” Struct. Control Health Monit. 23 (12): 1405–1416. https://doi.org/10.1002/stc.1850.
Yoon, H., V. Hoskere, J.-W. Park, and B. Spencer. 2017. “Cross-correlation-based structural system identification using unmanned aerial vehicles.” Sensors 17 (9): 2075. https://doi.org/10.3390/s17092075.
Zappa, E., P. Mazzoleni, and A. Matinmanesh. 2014. “Uncertainty assessment of digital image correlation method in dynamic applications.” Opt. Lasers Eng. 56: 140–151. https://doi.org/10.1016/j.optlaseng.2013.12.016.
Information & Authors
Information
Published In
Journal of Structural Engineering
Volume 145 • Issue 7 • July 2019
Copyright
©2019 American Society of Civil Engineers.
History
Received: Feb 13, 2018
Accepted: Oct 29, 2018
Published online: May 6, 2019
Published in print: Jul 1, 2019
Discussion open until: Oct 6, 2019
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.