A Review of Data Management and Visualization Techniques for Structural Health Monitoring Using BIM and Virtual or Augmented Reality
Publication: Journal of Structural Engineering
Volume 149, Issue 1
Abstract
Most civil infrastructure worldwide is currently past its design life. This situation has precipitated the need for more systematic inspections and continual monitoring of infrastructure to assure structural integrity. With the advancement of sensing and computing technologies, rapid, remote, real-time, and robust structural health monitoring (SHM) techniques have been significantly developed. These smart SHM technologies often result in big data that require advanced data management, visualization, diagnostic, and prognostic techniques. Over the past few decades, researchers have developed numerous machine learning and artificial intelligence (AI)–based damage diagnostic and prognosis methods, which have been systematically reviewed in the recent state-of-the-art papers. In parallel, various data management and visualization techniques have been explored in SHM using building information modeling (BIM), virtual reality (VR), and augmented reality (AR). Both BIM and AR/VR present a unique opportunity to document, systematically interpret, and visualize SHM data in a three-dimensional (3D) environment and have shown significant promise in a wide range of infrastructure monitoring applications. Unlike diagnostics and prognostics methods of SHM, there has been a very limited systematic review of the latest visualization and data management techniques, which are both the objective and novelty of this review paper. Each subtopic reviews The recent data management and visualization technologies, their relevance to SHM, and their implementation challenges in broader structural engineering applications. Finally, this state-of-the-art review summarizes potential future research directions for BIM and VR/AR associated with SHM.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
No data, models, or code were generated or used during the study.
Acknowledgments
This research was funded through the Natural Sciences and Engineering Research Council of Canada Western Strategic Support (NSERC WSS) Accelerator provided by Research Western to the corresponding author. The second author of this paper was awarded an NSERC Undergraduate Student Research Award, which is gratefully acknowledged.
References
Agnisarman, S., S. Lopes, K. C. Madathil, K. Piratla, and A. Gramopadhye. 2019. “A survey of automation-enabled human-in-the-loop systems for infrastructure visual inspection.” Autom. Constr. 97 (May): 52–76. https://doi.org/10.1016/j.autcon.2018.10.019.
Aguero, M., D. Maharjan, M. Rodriguez, D. Mascarenas, and F. Moreu. 2020. “Design and implementation of a connection between augmented reality and sensors.” Robotics 9 (1): 3. https://doi.org/10.3390/robotics9010003.
Ahmed, S. 2019. “A review on using opportunities of augmented reality and virtual reality in construction project management.” Organ. Technol. Manage. Constr. 11 (Aug): 1839–1852. https://doi.org/10.2478/otmcj-2018-0012.
Aires, M., M. López-Alonso, and M. Martinez-Rojas. 2018. “building information modeling and safety management: A systematic review.” Saf. Sci. 101 (Jun): 11–18. https://doi.org/10.1016/j.ssci.2017.08.015.
Angulo, C., K. Díaz, J. M. Cutiérrez, A. Prado, R. Casadey, G. Pannillo, F. Muñoz-La Rivera, R. F. Herrera, and J. C. Vielma. 2020. “Using BIM for the assessment of the seismic performance of educational buildings.” Int. J. Saf. Secur. Eng. 10 (1): 77–82. https://doi.org/10.18280/ijsse.100110.
Anil, E. B., B. Akinci, Ö. Kurç, and J. H. Garrett. 2016. “Building-information modeling-based earthquake damage assessment for reinforced concrete walls.” J. Comput. Civ. Eng. 30 (Jun): 04015076. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000551.
Artus, M., and C. Koch. 2019. “State of the art in damage information modeling for bridges.” In Proc., 26th Int. Workshop on Intelligent Computing in Engineering. Plymouth, UK: European Group for Intelligent Computing in Engineering.
ASCE. 2021. “2021 report card for America’s infrastructure.” Accessed August 31, 2022. https://infrastructurereportcard.org/.
Attard, L., C. Debono, and G. Valentino. 2018. “A comprehensive virtual reality system for tunnel surface documentation and structural health monitoring.” In Proc., IEEE Int. Conf. on Imaging Systems and Techniques. New York: IEEE. https://doi.org/10.1109/IST.2018.8577139.
Avci, O., O. Abdeljaber, S. Kiranyaz, M. Hussein, M. Gabbouj, and D. Inman. 2021. “A review of vibration-based damage detection in civil structures: From traditional methods to machine learning and deep learning application.” Mech. Syst. Sig. Process. 147 (Mar): 107077. https://doi.org/10.1016/j.ymssp.2020.107077.
Azimi, M., A. D. Eslamlou, and G. Pekcan. 2020. “Data-driven structural health monitoring and damage detection through deep learning: State-of-the-art review.” Sensors 20 (10): 2778. https://doi.org/10.3390/s20102778.
Aziz, Z., Z. Riaz, and M. Arslan. 2017. “Leveraging BIM and big data to deliver well maintained highways.” Facilities 35 (13–14): 818–832. https://doi.org/10.1108/F-02-2016-0021.
Bacco, M., P. Barsocchi, P. Cassará, D. Germanese, A. Gotta, G. Leone, D. Moroni, M. Pascali, and M. Tampucci. 2020. “Monitoring ancient buildings: Real development of an IoT system enhanced by UAVs and Virtual reality.” IEEE Accesss 8: 50131–50148. https://doi.org/10.1109/ACCESS.2020.2980359.
Badias, A., I. Alfaro, D. Gonzalez, F. Chinesta, and E. Cueto. 2018. “Reduced-order modelling for physically-based augmented reality.” Comput. Methods Appl. Mech. Eng. 341 (Mar): 53–70. https://doi.org/10.1016/j.cma.2018.06.011.
Bae, H., M. Golparvar-Fard, and J. White. 2013. “High-precision vision-based mobile augmented reality system for context-aware architectural, engineering, constructions and facility management (AEC/FM) applications.” Visualization Eng. 1 (1): 1–13. https://doi.org/10.1186/2213-7459-1-3.
Banfi, F., L. Barazzetti, M. Previtali, and F. Roncoroni. 2017. “Historic BIM: A new repository for structural health monitoring.” ISPRS—Int. Arch. Photogramm. Rem. Sens. Spatial Inf. Sci. 42W4 (Jun): 269–274. https://doi.org/10.5194/isprs-archives-XLII-5-W1-269-2017.
Barbosh, M., P. Singh, and A. Sadhu. 2021. “Empirical model decomposition and its variants: A review with applications in structural health monitoring.” Smart Mater. Struct. 29 (9): 093001. https://doi.org/10.1088/1361-665X/aba539.
Baroroh, D. K., C. H. Chu, and L. Wang. 2021. “Systematic literature review on augmented reality in smart manufacturing: Collaboration between human and computational intelligence.” J. Manuf. Syst. 61 (Oct): 696–711. https://doi.org/10.1016/j.jmsy.2020.10.017.
Behzadan, A., S. Dong, and V. Kamat. 2015. “Augmented reality visualization: A review of civil infrastructure system applications.” Adv. Eng. Inf. 29 (2): 252–267. https://doi.org/10.1016/j.aei.2015.03.005.
Behzadan, A., B. Timm, and V. Kamat. 2008. “General-purpose modular hardware and software framework for mobile outdoor augmented reality applications in engineering.” Adv. Eng. Inf. 22 (1): 90–105. https://doi.org/10.1016/j.aei.2007.08.005.
Berkenmeier, L., B. Zobel, S. Werning, I. Ickerott, and O. Thomas. 2019. “Engineering of augmented reality-based information systems.” Bus. Inf. Syst. Eng. 61 (1): 67–89. https://doi.org/10.1007/s12599-019-00575-6.
Bernasconi, A., M. Kharshiduzzman, L. F. Anodio, M. Bordegoni, G. M. Re, F. Braghin, and L. Comolli. 2014. “Development of a monitoring system for crack growth in bonded single-lap joints based on the strain field and visualization by augmented reality.” J. Adhes. 90 (5–6): 496–510. https://doi.org/10.1080/00218464.2013.857606.
Biringer, B. E., E. D. Vugrin, and D. E. Warren. 2013. Critical infrastructure system security and resiliency. Boca Raton, FL: CRC Press.
Boddupalli, C., A. Sadhu, and E. Azar. 2018. “An integrated structural health monitoring tool using building information modeling.” In Proc., CSCE Conf. Montreal: Canadian Society for Civil Engineering.
Boddupalli, C., A. Sadhu, E. Azar, and S. Pattyson. 2019. “Improved visualization of infrastructure monitoring data using building information modeling.” Struct. Infrastruct. Eng. 15 (9): 1247–1263. https://doi.org/10.1080/15732479.2019.1602150.
Bouchlaghem, D., H. Shang, J. Whyte, and A. Ganah. 2005. “Visualization in architecture, engineering and construction (AEC).” Autom. Constr. 14 (Jun): 287–295. https://doi.org/10.1016/j.autcon.2004.08.012.
Bruno, S., M. Fino, and F. Fatiguso. 2017. “Historic building information modelling: Performance assessment for diagnosis-aided information modelling and management.” Autom. Constr. 86 (Feb): 256–276. https://doi.org/10.1016/j.autcon.2017.11.009.
Caterino, N., I. Nuzzo, A. Ianniello, G. Varchetta, and E. Cosenza. 2021. “A BIM-based decision making framework for optimal seismic retrofit of existing buildings.” Eng. Struct. 242 (Sep): 112544. https://doi.org/10.1016/j.engstruct.2021.112544.
Chacón, R., F. Claure, and O. de Coss. 2020. “Development of VR/AR applications for experimental tests of beams, columns, and frames.” J. Comput. Civ. Eng. 34 (5): 05020003 https://doi.org/10.1061/%28ASCE%29CP.1943-5487.0000908.
Chen, A. Y., and J. C. Chu. 2016. “TDVRP and BIM integrated approach for in-building emergency rescue routing.” J. Comput. Eng. 30 (5): C4015003. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000522.
Chen, K., W. Lu, Y. Peng, S. Rowlinson, and G. Q. Huang. 2015. “Bridging BIM and building: From a literature review to an integrated conceptual framework.” Int. J. Project Manage. 33 (6): 1405–1416. https://doi.org/10.1016/j.ijproman.2015.03.006.
Chi, H. L., S. C. Kang, and X. Wang. 2013. “Research trends and opportunities of augmented reality applications in architecture, engineering, and construction.” Autom. Constr. 33 (Jun): 116–122. https://doi.org/10.1016/j.autcon.2012.12.017.
Dabous, S., and S. Feroz. 2020. “Condition monitoring of bridges with noncontact testing technologies.” Autom. Constr. 116 (Mar): 103224. https://doi.org/10.1016/j.autcon.2020.103224.
Dai, F., M. Lu, and V. R. Kamat. 2011. “Analytical approach to augmenting site photos with 3D graphics of underground infrastructure in construction engineering applications.” J. Comput. Civ. Eng. 25 (1): 66–74. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000072.
Dang, N. S., H. R. Kang, S. Lon, and C. S. Shim. 2018. “3D digital twin models for bridge maintenance.” In Proc., 10th Int. Conf. on Short and Medium Span Bridges. Seoul: Chung-Ang Univ.
Dang, N. S., and C. S. Shim. 2020. “BIM-based innovative bridge maintenance system using augmented reality technology.” Innovation Sustainable Infrastruct. 54 (Apr): 1217–1222. https://doi.org/10.1007/978-981-15-0802-8_195.
Delgado, J. M. D., I. Brilakis, and C. Middleton. 2016a. “Modelling, management, and visualisation of structural performance monitoring data on BIM.” In Proc., Int. Conf. on Smart Infrastructure and Construction. London: ICE Publishing.
Delgado, J. M. D., L. J. Butler, I. Brilakis, M. Z. E. B. Elshafie, and C. R. Middleton. 2018. “Structural performance monitoring using a dynamic data-driven BIM environment.” J. Comput. Civ. Eng. 32 (3): 04018009. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000749.
Delgado, J. M. D., L. J. Butler, N. Gibbons, I. Brilakis, M. Elshafie, and C. Middleton. 2016b. “Management of structural monitoring data of bridges using BIM.” Inst. Civ. Eng. 170 (3): 204–218. https://doi.org/10.17863/CAM.6377.
Delgado, J. M. D., L. Oyedele, P. Demian, and T. Beach. 2020. “A research agenda for augmented and virtual reality in architecture, engineering, and construction.” Adv. Eng. Inf. 45 (Aug): 101122. https://doi.org/10.1016/j.aei.2020.101122.
Del Grosso, A., P. Basso, L. Ruffini, F. Figini, and M. Cademartori. 2017. “Infrastructure management integrating SHM and BIM procedures.” In Proc., Fourth Conf. on Smart Monitoring, Assessment and Rehabilitation of Civil Structures. Cambridge, UK: Univ. of Cambridge.
De Marchi, L., A. Ceruti, A. Marzani, and A. Liverani. 2013. “Augmented reality to support on-field post-impact maintenance operations on thin structures.” J. Sens. 2013: 619570. https://doi.org/10.1155/2013/619570.
Deng, L., S. Lai, J. Ma, L. Lei, M. Zhong, L. Liao, and Z. Zhou. 2021. “Visualization and monitoring information management of bridge structure health and safety early warning based on BIM.” J. Asian Archit. Build. Eng. 21 (2): 427–438. https://doi.org/10.1080/13467581.2020.1869013.
Doghri, W., A. Saddoud, and L. C. Fourati. 2021. “Cyber-physical systems for structural health monitoring: Sensing technologies and intelligent computing.” J. Supercomputing 78 (1): 766–809. https://doi.org/10.1007/s11227-021-03875-5.
Dong, C., and F. Catbas. 2021. “A review of computer vision-based structural health monitoring at local and global levels.” Struct. Health Monit. 20 (2): 692–743. https://doi.org/10.1177/1475921720935585.
Dong, S., C. Feng, and V. Kamat. 2013. “Sensitivity analysis of augmented reality-assisted building damage reconnaissance using virtual prototyping.” Autom. Constr. 33 (Jan): 24–36. https://doi.org/10.1016/j.autcon.2012.09.005.
Dunphy, K., and A. Sadhu. 2022. “Autonomous crack detection approach for masonry structure using artificial intelligence.” In Recent developments in structural health monitoring and assessment–opportunities and challenges: Bridges, buildings and other infrastructures, 253–283. Singapore: World Scientific.
Dunston, P., and X. Wang. 2005. “Mixed reality-based visualization interfaces for architecture, engineering, and construction industry.” J. Constr. Eng. Manage. 131: 12. https://doi.org/10.1061/(ASCE)0733-9364(2005)131:12(1301).
Erickson, A., K. Kangsoo, B. Gerd, and F. W. Gregory. 2020. “Exploring the limitations of environment lighting on optical see-through head-mounted displays.” In Proc., Symp. on Spatial User Interaction, 1–8. New York: Association for Computing Machinery.
Fai, S., K. Graham, T. Duckworth, N. Wood, and R. Attar. 2014. “Building information modelling and heritage documentation.” In Proc., 23rd Int. Symp., Int. Scientific Committee for Documentation of Cultural Heritage (CIPA). Ottawa: Carleton Univ.
Feng, D., and M. Feng. 2018. “Computer vision for SHM of civil infrastructure: From dynamic response measurement to damage detection—A review.” Eng. Struct. 156 (Feb): 105–117. https://doi.org/10.1016/j.engstruct.2017.11.018.
Filardo, M. M., T. Walther, S. Maddineni, and H. J. Bargstädt. 2021. “Installing reinforcement rebars using virtual reality and 4D visualization.” In Computing in civil and building engineering, 1200–1216. Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-030-51295-8_84.
Gabbard, J. L., S. Edward, and D. Hix. 2006. “The effects of text drawing styles, background textures, and natural lighting on text legibility in outdoor augmented reality.” Presence 15 (1): 16–32. https://doi.org/10.1162/pres.2006.15.1.16.
Glisic, B., M. Yarnold, F. Moon, and A. Aktan. 2014. “Advanced visualization and accessibility to heterogenous monitoring data.” Comput.-Aided Civ. Infrastruct. Eng. 29 (5): 382–398. https://doi.org/10.1111/mice.12060.
Gulgec, N., G. Shahidi, T. Matarazzo, and S. Pakzad. 2017. “Current challenges with big data analytics in structural health monitoring.” In Proc., Int. Modal Analysis Conf. Bethlehem, PA: Lehigh Univ.
Hammad, A., H. Wang, and S. P. Mudur. 2009. “Distributed augmented reality for visualizing collaborative construction tasks.” J. Comput. Civ. Eng. 23 (6): 418–427. https://doi.org/10.1061/(ASCE)0887-3801(2009)23:6(418).
Hazra, B., A. Sadhu, A. Roffel, and S. Narasimhan. 2012. “Hybrid time-frequency blind source separation towards ambient system identification.” Comput.-Aided Civ. Infrastruct. Eng. 27 (5): 314–332. https://doi.org/10.1111/j.1467-8667.2011.00732.x.
Hou, G., L. Li, Z. Xu, Q. Chen, Y. Liu, and B. Qiu. 2019. “A BIM-based visual warning management system for structural health monitoring integrated with LSTM network.” KSCE J. Civ. Eng. 25 (8): 2779–2793. https://doi.org/10.1007/s12205-021-0565-0.
Huang, J. M., S. K. Ong, and A. Y. C. Nee. 2017. “Visualization and interaction of finite element analysis in augmented reality.” Comput.-Aided Des. 84 (Mar): 1–14. https://doi.org/10.1016/j.cad.2016.10.004.
Insa-Iglesias, M., M. D. Jenkins, and G. Morison. 2021. “3D visual inspection system framework for structural condition monitoring and analysis.” Autom. Constr. 128 (Aug): 103755. https://doi.org/10.1016/j.autcon.2021.103755.
Jeong, S., R. Hou, J. P. Lynch, H. Sohn, and K. H. Law. 2017. “An information modeling framework for bridge monitoring.” Adv. Eng. Software 114 (Dec): 11–31. https://doi.org/10.1016/j.advengsoft.2017.05.009.
Jeong, S., Y. Zhang, S. O’Connor, J. P. Lynch, H. Sohn, and K. H. Law. 2016. “A NoSQL data management infrastructure for bridge monitoring.” Smart Struct. Syst. 17 (4): 669–690. https://doi.org/10.12989/sss.2016.17.4.669.
Jiang, F., L. Ma, T. Broyd, and K. Chen. 2021. “Digital twins and its implementation in the civil engineering sector.” Autom. Constr. 130 (4): 103838. https://doi.org/10.1016/j.autcon.2021.103838.
Kaartinen, E., K. Dunphy, and A. Sadhu. 2022. “LiDAR-based structural health monitoring: Applications in civil infrastructure systems.” Sensors 22 (12): 4610. https://doi.org/10.3390/s22124610.
Kaewunruen, S., J. Sresakoolchai, W. Ma, and O. Phil-Ebosie. 2021. “Digital twin aided vulnerability assessment and risk-based maintenance planning of bridge infrastructures exposed to extreme conditions.” Sustainability 13 (4): 2051. https://doi.org/10.3390/su13042051.
Kamat, V. R., and S. El-Tawil. 2007. “Evaluation of augmented reality for rapid assessment of earthquake-induced building damage.” J. Comput. Civ. Eng. 21 (5): 303–310. https://doi.org/10.1061/(ASCE)0887-3801(2007)21:5(303).
Karaaslan, E., U. Bagci, and F. N. Catbas. 2019. “Artificial intelligence assisted infrastructure assessment using mixed reality systems.” Transp. Res. Rec. 2673 (12): 413–424. https://doi.org/10.1177/0361198119839988.
Kilic, G., and A. Caner. 2020. “Augmented reality for bridge condition assessment using advanced non-destructive techniques.” Struct. Infrastruct. Eng. 17 (7): 977–989. https://doi.org/10.1080/15732479.2020.1782947.
Kim, B., C. Kim, and H. Kim. 2012. “Interactive modeler for construction equipment operation using augmented reality.” J. Comput. Civ. Eng. 26 (3): 331–341. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000137.
Kim, B., and J. Kim. 2021. “BIM to AR matching technology of building maintenance platform using 5G-Based AR.” Preprint, submitted on May 24, 2021. https://www.preprints.org/manuscript/202105.0535/v1.
Kim, M. J., X. Wang, H. Li, and S. C. Kang. 2013. “Virtual reality for the built environment: A critical review of recent advances.” J. Inf. Technol. Constr. 18 (2013): 279–305.
Li, H., T. Yi, L. Ren, D. Li, and L. Huo. 2014. “Reviews on innovations and application in structural health monitoring.” Struct. Monit. Maint. 1 (1): 1–45. https://doi.org/10.12989/smm.2014.1.1.001.
Li, W., A. Nee, and S. Ong. 2017. “A state-of-the-art review of augmented reality in engineering analysis and simulation.” Multimodal Technol. Interact. 1 (3): 17. https://doi.org/10.3390/mti1030017.
Lin, T. 2017. “Virtual reality of earthquake ground motions for emergency response.” In Proc., 6th Int. Conf. on Computational Methods in Structural Dynamics and Earthquake Engineering, 2089–2094. Milwaukee, WI: Marquette Univ. https://doi.org/10.7712/120117.5551.18465.
Lin, T.-H., C.-H. Liu, M.-H. Tsai, and S.-C. Kang. 2015. “Using augmented reality in a multiscreen environment for construction discussion.” J. Comput. Civ. Eng. 29 (6): 04014088. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000420.
Liu, F., and S. Seipel. 2015. “Infrared-visible image registration for augmented reality-based thermographic building diagnostics.” Visualization Eng. 3 (16): 1–15. https://doi.org/10.1186/s40327-015-0028-0.
Ma, L., R. Sacks, and R. Zeibak-Shini. 2015. “Information modeling of earthquake-damaged reinforced concrete structures.” Adv. Eng. Inf. 29 (3): 396–407. https://doi.org/10.1016/j.aei.2015.01.007.
Ma, X., Y. Liu, J. Wang, D. Wang, and Y. Liu. 2020. “Application of the virtual reality technology in structural safety monitoring of shield tunnels.” In Proc., IOP Conf. Series: Materials Science and Engineering—5th World Multidisciplinary Civil Engineering-Architecture-Urban Planning Symp. Bristol, UK: IOP Publishing.
Madsen, C. B., T. Jensen, and M. Andersen. 2006. “Real-time image-based lighting for outdoor augmented reality under dynamically changing illumination conditions.” In Proc., Int. Conf. on Graphics Theory and Applications, 364–371. Aalborg, Denmark: Aalborg Univ.
Maharjan, D., M. Agüero, D. Mascarenas, R. Fierro, and F. Moreu. 2020. “Enabling human-infrastructure interfaces for inspection using augmented reality.” Struct. Health Monit. 20 (4): 1980–1996. https://doi.org/10.1177/1475921720977017.
Malekloo, A., E. Ozer, M. Al Hamaydeh, and M. Girolami. 2021. “Machine learning and structural health monitoring overview with emerging technology and high-dimensional data source highlights.” Struct. Health Monit. 21 (4): 1906–1955. https://doi.org/10.1177/14759217211036880.
Mallela, J., K. Gilson, and P. Goodrum. 2020. Leveraging augmented reality for highway construction. Washington, DC: Federal Highway Administration.
Manino, A., M. C. Dejaco, and F. R. Cecconi. 2021. “Building information modeling and internet of things integration for facility management—Literature review and future needs.” Appl. Sci. 11 (7): 3062. https://doi.org/10.3390/app11073062.
Martínez, H., S. Laukkanen, and J. Mattila. 2014. “A new flexible augmented reality platform for development of maintenance and educational applications.” Int. J. Virtual Works Hum. Comput. Interact. 2: 18–27. https://doi.org/10.11159/vwhci.2014.003.
Marzouk, M., and M. Hisham. 2012. “Bridge information modeling in sustainable bridge management.” In Proc., Int. Conf. on Sustainable Design and Construction (ICSDC), 457–466. Reston, VA: ASCE. https://doi.org/10.1061/41204(426)57.
Marzouk, M. M., M. Hisham, and K. Al-Gahtani. 2014. “Applications of bridge information modelling in bridges life cycle.” Smart Struct. Syst. 13 (3): 407–418. https://doi.org/10.12989/sss.2014.13.3.407.
Mascarenas, D., et al. 2019. “Augmented reality for enabling smart nuclear infrastructure.” Front. Built Environ. 5 (Jun): 82. https://doi.org/10.3389/fbuil.2019.00082.
Mascareñas, D. D., et al. 2020. “Augmented reality for next-generation infrastructure inspections.” Struct. Health Monit. 20 (4): 1957–1979. https://doi.org/10.1177/1475921720953846.
McGuire, B., R. Atadeo, C. Clevenger, and M. Ozbek. 2016. “Bridge information modeling for inspection and evaluation.” J. Bridge Eng. 21 (4): 04015076. https://doi.org/10.1061/(ASCE)BE.1943-5592.0000850.
Moreu, F., B. Bleck, S. Vemuganti, D. Rogers, and D. Mascarenas. 2017. “Augmented reality tools for enhanced structural inspection.” Struct. Health Monit. https://doi.org/10.12783/shm2017/14221.
Muin, S., and K. M. Mosalam. 2021. “Human-machine collaboration framework for structural health monitoring and resiliency.” Eng. Struct. 235 (May): 112084. https://doi.org/10.1016/j.engstruct.2021.112084.
Musella, C., M. Serra, C. Menna, and D. Asprone. 2021. “Building information modeling and artificial intelligence: Advanced technologies for the digitalisation of seismic damage in existing buildings.” Struct. Concr. 22: 2761–2774. https://doi.org/10.1002/suco.202000029.
Napolitano, R., A. Blyth, and B. Glisic. 2017a. “Virtual environments for structural health monitoring.” Struct. Health Monit. https://doi.org/10.12783/shm2017/14031.
Napolitano, R., A. Blyth, and B. Glisic. 2018. “Virtual environments for visualizing structural health monitoring sensor networks, data, and metadata.” Sensors 18 (1): 243. https://doi.org/10.3390/s18010243.
Napolitano, R., Z. Liu, C. Sun, and B. Glisic. 2019. “Combination of image-based documentation and augmented reality for structural health monitoring and building pathology.” Front. Built Environ. 5 (Apr): 50. https://doi.org/10.3389/fbuil.2019.00050.
Napolitano, R., G. Scherer, and B. Glisic. 2017b. “Virtual tours and informational modeling for conservation of cultural heritage sites.” J. Cult. Heritage 29 (Jan): 123–129. https://doi.org/10.1016/j.culher.2017.08.007.
Oreto, C., S. A. Biancardo, N. Viscione, R. Veropalumbo, and F. Russo. 2021. “Road pavement information modeling through maintenance scenario evaluation.” J. Adv. Transp. 14 (May): 1–14. https://doi.org/10.1155/2021/8823117.
O’Shea, M., and J. Murphy. 2020. “Design of a BIM integrated structural health monitoring system for a historic offshore lighthouse.” Buildings 10 (7): 131. https://doi.org/10.3390/buildings10070131.
Palmarini, R., J. A. Erokiyuncu, R. Roy, and H. Torabmostaedi. 2018. “A systematic review of augmented reality applications in maintenance.” Rob. Comput. Integr. Manuf. 49 (Feb): 215–228. https://doi.org/10.1016/j.rcim.2017.06.002.
Panah, R., and M. Kioumarsi. 2021. “Application of building information modeling in the health monitoring and maintenance process: A systematic review.” Sensors 21 (3): 837. https://doi.org/10.3390/s21030837.
Peng, Z. 2020. “An operation and maintenance strategy of intelligent building information model data based on cloud computing.” Int. Inf. Eng. Technol. Assoc. 25 (4): 461–467. https://doi.org/10.18280/isi.250408.
Rankohi, S., and L. Waugh. 2013. “Review and analysis of augmented reality literature for construction industry.” Visualization Eng. 1 (9): 1–8. https://doi.org/10.1186/2213-7459-1-9.
Ranyal, E., A. Sadhu, and K. Jain. 2022. “Road condition monitoring using smart sensing and artificial intelligence.” Sensors 22 (8): 3044. https://doi.org/10.3390/s22083044.
Sadhu, A., S. Narasimhan, and J. Antoni. 2017. “A review of output-only structural mode identification literature employing blind source separation.” Mech. Syst. Sig. Process. 94 (Sep): 415–431. https://doi.org/10.1016/j.ymssp.2017.03.001.
Sampaio, A., and O. Martins. 2014. “The application of virtual reality technology in the construction of bridges: The cantilever and incremental launching methods.” Autom. Constr. 37 (Jan): 58–67. https://doi.org/10.1016/j.autcon.2013.10.015.
Schall, G., E. Mendez, E. Kruijff, E. Veas, S. Junghanns, B. Reitinger, and D. Schmalstieg. 2009. “Handheld augmented reality for underground infrastructure visualization.” Personal Ubiquitous Comput. 13 (4): 281–291. https://doi.org/10.1007/s00779-008-0204-5.
Shin, D. H., and P. S. Dunston. 2008. “Identification of application areas for augmented reality in industrial construction based on technology suitability.” Autom. Constr. 17 (7): 882–894. https://doi.org/10.1016/j.autcon.2008.02.012.
Shin, D. H., and P. S. Dunston. 2009. “Evaluation of augmented reality in steel column inspection.” Autom. Constr. 18 (2): 118–129. https://doi.org/10.1016/j.autcon.2008.05.007.
Shirazi, A., and A. H. Behzadan. 2014. “Design and assessment of mobile augmented reality-based information delivery tool for construction and civil engineering curriculum.” J. Civ. Eng. Educ. 141 (3): 04014012. https://doi.org/10.1061/(ASCE)EI.1943-5541.0000229.
Sidani, A., F. Dinis, J. Duarte, L. Sanhudo, D. Calvetti, J. Baptista, J. Martins, and A. Soeiro. 2021. “Recent tools and techniques of BIM-based augmented reality: A systematic review.” J. Build. Eng. 42 (Oct): 102500. https://doi.org/10.1016/j.jobe.2021.102500.
Singh, P., M. Keyvanlou, and A. Sadhu. 2021. “An improved time-varying empirical mode decomposition for structural condition assessment using limited sensors.” Eng. Struct. 232 (Aug): 111882. https://doi.org/10.1016/j.engstruct.2021.111882.
Singh, P., and A. Sadhu. 2020. “System identification-enhanced visualization tool for infrastructure monitoring and maintenance.” Front. Built Environ. 6: 76. https://doi.org/10.3389/fbuil.2020.00076.
Smarsly, K., M. Theiler, and K. Dragos. 2017. “BIM-based design of structural health monitoring systems.” In Proc., 11th Int. Workshop on Structural Health Monitoring. Hamburg, Germany: Technische Universität Hamburg. https://doi.org/10.12783/shm2017/13941.
Sony, S., K. Dunphy, A. Sadhu, and M. Capretz. 2021. “A systematic review of convolutional neural network-based structural condition assessment techniques.” Eng. Struct. 226 (Jun): 111347. https://doi.org/10.1016/j.engstruct.2020.111347.
Sony, S., S. Gamage, A. Sadhu, and J. Samarabandu. 2022. “Multiclass damage identification in a full-scale bridge using optimally tuned one-dimesional convolutional neural network.” J. Comput. Civ. Eng. 36 (2): 04021035. https://doi.org/10.1061/(ASCE)CP.1943-5487.0001003.
Sony, S., S. Laventure, and A. Sadhu. 2019. “A literature review of next-generation smart sensing technology in structural health monitoring.” Struct. Control Health Monit. 26 (3): e2321. https://doi.org/10.1002/stc.2321.
Spencer, B. F., V. Hoskere, and Y. Narazaki. 2019. “Advances in computer vision-based civil infrastructure inspection and monitoring.” Engineering 5 (2): 199–222. https://doi.org/10.1016/j.eng.2018.11.030.
Sun, S., Z. Shang, Y. Xia, S. Bhowmick, and S. Nagarajaiah. 2020. “Review of bridge structural health monitoring aided by big data and artificial intelligence: From condition assessment to damage detection.” J. Struct. Eng. 146 (5): 04020073. https://doi.org/10.1061/(ASCE)ST.1943-541X.0002535.
Sutherland, I. E. 1968. “A head-mounted three-dimensional display.” In Proc., Fall Joint Computer Conf., Part I, 757–764. Salt Lake City: Univ. of Utah.
Syed, A. S., D. Sierra-Sosa, A. Kumar, and A. El Mamaghraby. 2021. “IoT in smart cities: A survey of technologies, practices and challenges.” Smart Cities 4 (2): 429–475. https://doi.org/10.3390/smartcities4020024.
Taha, M. R., B. M. Ayyub, K. Soga, S. Daghash, D. H. Murcia, F. Moreu, and E. Soliman. 2021. “Emerging technologies for resilient infrastructure: Conspectus and roadmap.” ASCE-ASME J. Risk Uncertainties Eng. Syst Part A: Civ. Eng. 7 (2): 03121002. https://doi.org/10.1061/AJRUA6.0001134.
Tan, Y., S. Li, H. Liu, P. Chen, and Z. Zhou. 2021. “Automatic inspection data collection of building surface based on BIM and UAV.” Autom. Constr. 131 (Jun): 103881. https://doi.org/10.1016/j.autcon.2021.103881.
Thabet, W., M. F. Shiratuddin, and D. Bowman. 2002. “Virtual reality in construction.” In Engineering computational technology, 25–52. Stirling, Scotland: Civil-Comp Press. https://doi.org/10.4203/csets.8.2.
Theiler, M., K. Dragos, and K. Smarsly. 2017. “BIM-based design of structural health monitoring systems.” In Proc., 11th Int. Workshop on Structural Health Monitoring. Stanford, CA: Stanford Univ.
Tsai, Y. H., J. Wang, W. T. Chien, C. Y. Wei, X. Wang, and S. H. Hsieh. 2019. “A BIM-based approach for predicting corrosion under insulation.” Autom. Constr. 107 (Nov): 102923. https://doi.org/10.1016/j.autcon.2019.102923.
Turkan, Y., R. Radkowski, A. Karabulut-Ilgu, A. Behzadan, and A. Chen. 2017. “Mobile augmented reality for teaching structural analysis.” Adv. Eng. Inf. 34 (Mar): 90–100. https://doi.org/10.1016/j.aei.2017.09.005.
Valinejadshoubi, M., A. Bagchi, and O. Moselhi. 2019. “Development of a BIM-based data management system for structural health monitoring with application to modular buildings: Case study.” J. Comput. Civ. Eng. 33 (3): 05019003. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000826.
Verde, S., M. Marcon, S. Milani, and S. Tubaro. 2020. “Advanced assistive maintenance based on augmented reality and 5G networking.” Sensors 20 (24): 7157. https://doi.org/10.3390/s20247157.
Vignali, V., E. M. Acerra, C. Lantieri, F. Di Vincenzo, G. Piacentini, and S. Pancalidi. 2021. “Building information modelling (BIM) application for existing road infrastructure.” Autom. Constr. 128 (Aug): 103752. https://doi.org/10.1016/j.autcon.2021.103752.
Vitiello, U., V. Ciotta, A. Salzano, D. Asprone, G. Manfredi, and E. Cosenza. 2019. “BIM-based approach for the cost-optimization of seismic retrofit startegies on existing building.” Autom. Constr. 98 (Feb): 90–101. https://doi.org/10.1016/j.autcon.2018.10.023.
Volk, R., F. Schultmann, and J. Stengel. 2014. “Building information modeling (BIM) for existing buildings—Literature review and future needs.” Autom. Constr. 38 (Aug): 109–127. https://doi.org/10.1016/j.autcon.2013.10.023.
Wang, J., Y. Fu, and X. Yang. 2017. “An integrated system for building structural health monitoring and early warning based on an internet of things approach.” Int. J. Distrib. Sens. Netw. 13 (1): 1550147716689101. https://doi.org/10.1177/1550147716689101.
Wang, P., P. Wu, J. Wang, H. Chi, and X. Wang. 2018a. “A critical review of the use of virtual reality in construction engineering education and training.” Int. J. Environ. Res. Public Health 15 (6): 1204. https://doi.org/10.3390/ijerph15061204.
Wang, S., R. Guo, H. Wang, Y. Ma, and Z. Zong. 2018b. “Manufacture assembly fault detection method based on deep learning and mixed reality.” In Proc., 2018 IEEE Int. Conf. on Information and Automation (ICIA), 808–813. New York: IEEE.
Wang, S., S. A. Zargar, and F. G. Yuan. 2020. “Augmented reality for enhanced visual inspection through knowledge-based deep learning.” Struct. Health Monit 20 (1): 426–442. https://doi.org/10.1177/1475921720976986.
Wang, X. 2007. “Using augmented reality to plan virtual construction worksite.” Int. J. Adv. Rob. Syst. 4 (4): 42–512. https://doi.org/10.5772/5677.
Wang, X., M. J. Kim, P. Love, and S. C. Kang. 2013. “Augmented reality in built environment: Classification and implications for future research.” Autom. Constr. 32 (Jun): 1–13. https://doi.org/10.1016/j.autcon.2012.11.021.
Xu, J., and F. Moreu. 2021. “A review of augmented reality applications in civil infrastructure during the 4th industrial revolution.” Front. Built-Up Environ. 7 (Jul): 640732. https://doi.org/10.3389/fbuil.2021.640732.
Yabuki, N., K. Miyashita, and T. Fukuda. 2011. “An invisible height evaluation system for building height regulation to preserve good landscapes using augmented reality.” Autom. Constr. 20 (3): 228–235. https://doi.org/10.1016/j.autcon.2010.08.003.
Yamaguchi, T., T. Shibuya, M. Kanda, and A. Yasojima. 2019. “Crack inspection support system for concrete structures using head-mounted display in mixed reality space.” In Proc., 2019 58th Annual Conf. of the Society of Instrument and Control Engineers of Japan (SICE), 791–796. New York: IEEE.
Yeh, K.-C., M.-H. Tsai, and S.-C. Kang. 2012. “On-site building information retrieval by using projection-based augmented reality.” J. Comput. Civ. Eng. 26 (3): 342–355. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000156.
Zhao, Y., X. Deng, and H. Lai. 2021. “Reconstructing BIM from 2D structural drawing for existing buildings.” Autom. Constr. 128 (Jul): 103750. https://doi.org/10.1016/j.autcon.2021.103750.
Information & Authors
Information
Published In
Journal of Structural Engineering
Volume 149 • Issue 1 • January 2023
Copyright
© 2022 American Society of Civil Engineers.
History
Published online: Oct 26, 2022
Published in print: Jan 1, 2023
Discussion open until: Mar 26, 2023
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Arabella Akman, Ayan Sadhu, Recent Development of 3D-Printing Technology in Construction Engineering, Practice Periodical on Structural Design and Construction, 10.1061/PPSCFX.SCENG-1278, 29, 1, (2024).
- Peng Li, Yuanjie Tang, Zhiming Zheng, Ziteng Wang, Yong Zhuang, Semiautomated Railway Line Information Modeling Based on Asset Management Data, Journal of Construction Engineering and Management, 10.1061/JCEMD4.COENG-15011, 150, 10, (2024).
- Alessandro Sabato, Shweta Dabetwar, Nitin Nagesh Kulkarni, Giancarlo Fortino, Noncontact Sensing Techniques for AI-Aided Structural Health Monitoring: A Systematic Review, IEEE Sensors Journal, 10.1109/JSEN.2023.3240092, 23, 5, (4672-4684), (2023).
- Omar Awadallah, Ayan Sadhu, Automated multiclass structural damage detection and quantification using augmented reality, Journal of Infrastructure Intelligence and Resilience, 10.1016/j.iintel.2022.100024, 2, 1, (100024), (2023).