GIS-Based Information System for Automated Building Façade Assessment Based on Unmanned Aerial Vehicles and Artificial Intelligence
Publication: Journal of Architectural Engineering
Volume 29, Issue 4
Abstract
Unmanned aerial vehicles (UAVs) have recently become popular in building façade inspections to maintain a safe and well-performed built environment. A camera-equipped UAV system can capture numerous high-resolution façade images for close-up visual inspections. However, in several cases, the multispectrum and spatiotemporal data collected by UAVs are not systematically documented and utilized, which obstructs the automation in the identification, localization, assessment, and tracking of façade anomalies. This paper develops an integrated, computational GIS-based information system to provide automated storage, retrieval, detection, assessment, and documentation of façade anomalies based on UAV-captured data. The developed system creates user-friendly access to diverse professional imagery analysis tools from external artificial intelligence (AI) algorithms. A real-world case was studied to present the procedure and advances in the management and analysis of multisourced inspection data to automate UAV-based façade diagnosis. As a result, the proposed method facilitates the seamless fusion, processing, visualization, and documentation of multimodal inspection data, resulting in convenient analysis with discrepancies measured in decimeters for length, millimeters for width, and centimeters for geoposition. This contributes to the understanding of façade conditions and decision-making of timely maintenance throughout a building’s service lifecycle.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
References
Abadi, M., et al. 2016. “TensorFlow: Large-scale machine learning on heterogeneous distributed systems.” https://doi.org/10.48550/arXiv.1603.0447
Alipour, M., D. K. Harris, and G. R. Miller. 2019. “Robust pixel-level crack detection using deep fully convolutional neural networks.” J. Comput. Civil Eng. 33 (6): 04019040. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000854.
Amaro, B., D. Saraiva, J. de Brito, and I. Flores-Colen. 2013. “Inspection and diagnosis system of ETICS on walls.” Constr. Build. Mater. 47 (Supplement C): 1257–1267. https://doi.org/10.1016/j.conbuildmat.2013.06.024.
Amaro, B., D. Saraiva, J. de Brito, and I. Flores-Colen. 2014. “Statistical survey of the pathology, diagnosis and rehabilitation of ETICS in walls.” J. Civ. Eng. Manage. 20 (4): 511–526. https://doi.org/10.3846/13923730.2013.801923.
Ariwoola, R. 2016. “Use of drone and infrared camera for a campus building envelope study.” M.S. thesis, Dept. of Technology, East Tennessee State Univ.
Bemis, S. P., S. Micklethwaite, D. Turner, M. R. James, S. Akciz, S. T. Thiele, and H. A. Bangash. 2014. “Ground-based and UAV-based photogrammetry: A multi-scale, high-resolution mapping tool for structural geology and paleoseismology.” J. Struct. Geol. 69: 163–178. https://doi.org/10.1016/j.jsg.2014.10.007.
Bonham-Carter, G. F. 2014. Geographic information systems for geoscientists: Modelling with GIS. Amsterdam, Netherlands: Elsevier.
Borrmann, D., A. Nüchter, M. Ðakulović, I. Maurović, I. Petrović, D. Osmanković, and J. Velagić. 2014. “A mobile robot based system for fully automated thermal 3D mapping.” Adv. Eng. Inf. 28 (4): 425–440. https://doi.org/10.1016/j.aei.2014.06.002.
Chen, K., and T. Rakha. 2021. “CV-based Registration of UAV-captured façade inspection images to 3D building point cloud models.” In Proc., Symp. on Simulation for Architecture and Urban Design SimAUD 2021. San Diego, CA: Society for Computer Simulation International.
Chen, K., G. Reichard, A. Akanmu, and X. Xu. 2021a. “Geo-registering UAV-captured close-range images to GIS-based spatial model for building façade inspections.” Autom. Constr. 122: 103503. https://doi.org/10.1016/j.autcon.2020.103503.
Chen, K., G. Reichard, X. Xu, and A. Akanmu. 2021b. “Automated crack segmentation in close-range building façade inspection images using deep learning techniques.” J. Build. Eng. 43: 102913. https://doi.org/10.1016/j.jobe.2021.102913.
Chen, K., X. Xu, G. Reichard, and A. Akanmu. 2021c. “An integrated computational GIS platform for UAV-based building façade inspection.” In EG-ICE 2021 Workshop on Intelligent Computing in Engineering. Berlin, Germany: Universitätsverlag of TU Berlin.
Costa, J., P. V. Paulo, F. A. Branco, and J. de Brito. 2014. “Modeling evolution of stains caused by collection of dirt in old building facades.” J. Perform. Constr. Facil 28 (2): 264–271. https://doi.org/10.1061/(ASCE)CF.1943-5509.0000417.
Culjak, I., D. Abram, T. Pribanic, H. Dzapo, and M. Cifrek. 2012. “A brief introduction to OpenCV.” In Proc., 35th Int. Convention on Information and Communication Technology, Electronics and Microelectronics MIPRO, 1725–1730. Piscataway, NJ: Institute of Electrical and Electronics Engineers (IEEE).
Dias, I., I. Flores-Colen, and A. Silva. 2021. “Critical analysis about emerging technologies for building’s façade inspection.” Buildings 11 (2): 53. https://doi.org/10.3390/buildings11020053.
Diebolt, K., J. Banta, and C. Corbin. 2004. “Direct digital input of façade survey data using handheld computing devices.” In Building facade maintenance, repair, and inspection, edited by J. Erdley, and T. Schwartz, 124. West Conshohocken, PA: ASTM.
Dore, C., and M. Murphy. 2012. “Integration of Historic Building Information Modeling (HBIM) and 3D GIS for recording and managing cultural heritage sites.” In Proc., 18th Int. Conf., on Virtual Systems and Multimedia, 369–376. Piscataway, NJ: Institute of Electrical and Electronics Engineers (IEEE).
Eltner, A., A. Kaiser, C. Castillo, G. Rock, F. Neugirg, and A. Abellán. 2016. “Image-based surface reconstruction in geomorphometry - merits, limits and developments.” Earth Surf. Dyn. 4 (2): 359–389. https://doi.org/10.5194/esurf-4-359-2016.
Eschmann, C., C.-M. Kuo, C.-H. Kuo, and C. Boller. 2012. “Unmanned aircraft systems for remote building inspection and monitoring.” In Proc., 6th European Workshop on Structural Health Monitoring. Dresden, Germany: NDT.net.
Fernandez Galarreta, J., N. Kerle, and M. Gerke. 2015. “UAV-based urban structural damage assessment using object-based image analysis and semantic reasoning.” Nat. Hazards Earth Syst. Sci. 15 (6): 1087–1101. https://doi.org/10.5194/nhess-15-1087-2015.
Flores-Colen, I., and J. de Brito. 2010. “A systematic approach for maintenance budgeting of buildings façades based on predictive and preventive strategies.” Constr. Build. Mater. 24 (9): 1718–1729. https://doi.org/10.1016/j.conbuildmat.2010.02.017.
Gaspar, K., M. Casals, and M. Gangolells. 2016. “Classifying system for façades and anomalies.” J. Perform. Constr. Facil 30 (1): 04014187. https://doi.org/10.1061/(ASCE)CF.1943-5509.0000693.
Guo, J., Q. Wang, Y. Li, and P. Liu. 2020. “Façade defects classification from imbalanced dataset using meta learning-based convolutional neural network.” Comput.-Aided Civ. Infrastruct. Eng. 35 (12): 1403–1418. https://doi.org/10.1111/mice.12578.
Ham, Y., and M. Golparvar-Fard. 2013. “An automated vision-based method for rapid 3D energy performance modeling of existing buildings using thermal and digital imagery.” Adv. Eng. Inf. 27 (3): 395–409. https://doi.org/10.1016/j.aei.2013.03.005.
Hoskere, V., Y. Narazaki, T. A. Hoang, and Jr B. F. Spencer. 2018. “Towards automated post-earthquake inspections with deep learning-based condition-aware models.” In Proc., 7th World Conf. on Structural Control and Monitoring. Harbin, China: Harbin Institute of Technology.
Hoskere, V., Y. Narazaki, T. Hoang, and Jr B. Spencer. 2017. “Vision-based structural inspection using multiscale deep convolutional neural networks.” In Proc., 3rd Huixian International Forum on Earthquake Engineering for Young Researchers. Urbana, IL: University of Illinois at Urbana-Champaign.
Kadiyala, A., and A. Kumar. 2017. “Applications of Python to evaluate environmental data science problems.” In Environ. Prog. Sustain Energy 36 (6): 1580–1586. https://doi.org/10.1002/ep.12786.
Kouzehgar, M., Y. Krishnasamy Tamilselvam, M. Vega Heredia, and M. Rajesh Elara. 2019. “Self-reconfigurable façade-cleaning robot equipped with deep-learning-based crack detection based on convolutional neural networks.” Autom. Constr. 108: 102959. https://doi.org/10.1016/j.autcon.2019.102959.
Lagüela, S., L. Díaz−Vilariño, D. Roca, and H. Lorenzo. 2015. “Aerial thermography from low-cost UAV for the generation of thermographic digital terrain models.” Opto-Electron. Rev. 23 (1): 78–84. https://doi.org/10.1515/oere-2015-0006.
Liu, Y., J. K. W. Yeoh, and D. K. H. Chua. 2020. “Deep learning–based enhancement of motion blurred UAV concrete crack images.” J. Comput. Civil Eng. 34 (5): 04020028. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000907.
Madureira, S., I. Flores-Colen, J. de Brito, and C. Pereira. 2017. “Maintenance planning of facades in current buildings.” Constr. Build. Mater. 147: 790–802. https://doi.org/10.1016/j.conbuildmat.2017.04.195.
Maset, E., A. Fusiello, F. Crosilla, R. Toldo, and D. Zorzetto. 2017. “Photogrammetric 3D building reconstruction from thermal images.” In Proc., Int. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 25–32. Bethesda, MD: International Society of Photogrammetry and Remote Sensing (ISPRS).
Mohan, A., and S. Poobal. 2018. “Crack detection using image processing: A critical review and analysis.” Alexandria Eng. J. 57: 787–798. https://doi.org/10.1016/j.aej.2017.01.020.
Murtiyoso, A., P. Grussenmeyer, N. Börlin, J. Vandermeerschen, and T. Freville. 2018. “Open source and independent methods for bundle adjustment assessment in close-range UAV photogrammetry.” Drones 2 (1): 3. https://doi.org/10.3390/drones2010003.
Pereira, F. C., and C. E. Pereira. 2015. “Embedded image processing systems for automatic recognition of cracks using UAVs.” IFAC-PapersOnLine 48 (10): 16–21. https://doi.org/10.1016/j.ifacol.2015.08.101.
Pires, R., J. de Brito, and B. Amaro. 2015. “Inspection, diagnosis, and rehabilitation system of painted rendered façades.” J. Perform. Constr. Facil 29 (2): 04014062. https://doi.org/10.1061/(ASCE)CF.1943-5509.0000534.
Puente, I., R. Lindenbergh, A. Van Natijne, R. Esposito, and R. Schipper. 2018. “Monitoring of progressive damage in buildings using laser scan data.” In Proc., Int. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 923–929. Bethesda, MD: International Society of Photogrammetry and Remote Sensing (ISPRS).
Rakha, T., A. Liberty, A. Gorodetsky, B. Kakillioglu, and S. Velipasalar. 2018. “Heat mapping drones: An autonomous computer-vision-based procedure for building envelope inspection using unmanned aerial systems (UAS).” Technol. Cancer Res. Treat. 2 (1): 30–44. https://doi.org/10.1080/24751448.2018.1420963.
Remondino, F., L. Barazzetti, F. Nex, M. Scaioni, and D. Sarazzi. 2011. “UAV photogrammetry for mapping and 3d modeling–current status and future perspectives.” Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 38 (1): C22.
Sá, G., J. Sá, J. de Brito, and B. Amaro. 2015. “Statistical survey on inspection, diagnosis and repair of wall renderings.” J. Civ. Eng. Manage. 21 (5): 623–636. https://doi.org/10.3846/13923730.2014.890666.
Sankarasrinivasan, S., E. Balasubramanian, K. Karthik, U. Chandrasekar, and R. Gupta. 2015. “Health monitoring of civil structures with integrated UAV and image processing system.” Procedia Comput. Sci. 54: 508–515. https://doi.org/10.1016/j.procs.2015.06.058.
Scaioni, M., E. Rosina, A. L’Erario, and L. Dìaz-Vilariño. 2017. “Integration of infrared thermography and photogrammetric surveying of built landscape.” In Proc., Int. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 153–160. Bethesda, MD: International Society of Photogrammetry and Remote Sensing (ISPRS).
Selvaraju, R. R., M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. 2020. “Grad-CAM: Visual explanations from deep networks via gradient-based localization.” Int. J. Comput. Vision 128 (2): 336–359. https://doi.org/10.1007/s11263-019-01228-7.
Shih, P.-H., and K.-H. Chi. 2020. “A deep learning application for detecting facade tile degradation.” In Proc., 2nd Int. Conf., on Human Systems Engineering and Design, edited by T. Ahram, W. Karwowski, S. Pickl, and R. Taiar, 26–32. Cham, Switzerland: Springer International Publishing. Advances in Intelligent Systems and Computing.
Silvestre, J. D., and J. de Brito. 2009. “Ceramic tiling inspection system.” Constr. Build. Mater. 23 (2): 653–668. https://doi.org/10.1016/j.conbuildmat.2008.02.007.
Spencer, B. F., V. Hoskere, and Y. Narazaki. 2019. “Advances in computer vision-based civil infrastructure inspection and monitoring.” Engineering 5 (2): 199–222. https://doi.org/10.1016/j.eng.2018.11.030.
Tateosian, L., and L. Tateosian. 2015. “Calling tools with arcpy.” In ArcGIS API for Python, 95–117. Cham, Switzerland: Springer International Publishing.
Vossoughi, H., and R. Siddiqui. 2004. “Industrial rope access—An alternate means for inspection, maintenance, and repair of building facades and structures.” In Building Facade Maintenance, Repair, and Inspection, edited by J. Erdley, and T. Schwartz, 116–123. West Conshohocken, PA: ASTM.
Wang, C., Y. K. Cho, and M. Gai. 2013. “As-is 3D thermal modeling for existing building envelopes using a hybrid LIDAR system.” J. Comput. Civil Eng. 27 (6): 645–656. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000273.
Wiss, Janney, Elstner Associates, Inc. 2020. “Facade ordinances.” Accessed December 12, 2020. http://www.facadeordinance.com/.
Yahyanejad, S., and B. Rinner. 2015. “A fast and mobile system for registration of low-altitude visual and thermal aerial images using multiple small-scale UAVs.” ISPRS J. Photogramm. Remote Sens. 104: 189–202. https://doi.org/10.1016/j.isprsjprs.2014.07.015.
Information & Authors
Information
Published In
Journal of Architectural Engineering
Volume 29 • Issue 4 • December 2023
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Feb 20, 2023
Accepted: Jun 26, 2023
Published online: Aug 16, 2023
Published in print: Dec 1, 2023
Discussion open until: Jan 16, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.