Abstract
After hazard events, large numbers of images are collected by reconnaissance teams to document the post-event state of structures, and to assess their performance and improve design procedures and codes. The majority of these data are captured as images and manually labeled. This highly repetitive task requires considerable domain expertise and time. Advances in deep learning have enabled researchers to rapidly classify reconnaissance images. Thus far, these classification methods are limited to a simple classification schema in which the classes are all either mutually exclusive or independent. To date, an efficient classification system of a complex schema containing many classes arranged in a multi-level hierarchical structure is not available to support earthquake reconnaissance. To address this gap, this paper introduces a comprehensive classification schema and a multi-output deep convolutional neural network (DCNN) model for rapid postearthquake image classification. In contrast to past work, herein a single multi-output DCNN classification model with a hierarchy-aware prediction was trained to enable the rapid organization of images. The performance of the proposed multi-output model was validated through comparisons with multi-label and multi-class models using an F1-score. As result, the multi-output model outperformed other models. Then, the multi-output model was deployed to a web-based platform called the Automated Reconnaissance Image Organizer, which can be used to easily organize earthquake reconnaissance images.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Reconnaissance images used for developing ARIO are publicly available at https://datacenterhub.org/. The source code for the multi-output image classification model will be shared through GitHub upon acceptance of the paper. The ground-truth labels of the images will be available from the corresponding author by request.
Acknowledgments
Our research team acknowledges the support from the Natural Sciences and Engineering Research Council of Canada under Grant No. RGPIN-2020-03979, the National Science Foundation under Grant No. NSF 1835473, and the valuable image contributions from the Center for Earthquake Engineering and Disaster Data at Purdue University.
References
ATC (Applied Technology Council). 1998. Evaluation of earthquake damaged concrete and masonry wall buildings. Washington, DC: Federal Emergency Management Agency.
ATC (Applied Technology Council). 2004. Vol. 45 of Field manual: Safety evaluation of buildings after windstorms and floods. 1st ed. Redwood City: Applied Technology Council.
ATC (Applied Technology Council). 2005. Field manual: Postearthquake safety evaluation of buildings. 2nd ed. Redwood City: Applied Technology Council.
ATC (Applied Technology Council). 2018. Seismic performance assessment of buildings. 2nd ed. Redwood City: Applied Technology Council.
Azimi, M., A. D. Eslamlou, and G. Pekcan. 2020. “Data-driven structural health monitoring and damage detection through deep leaning: State-of-the-art review.” Sensors 20 (10): 2778. https://doi.org/10.3390/s20102778.
Bray, J. D., J. D. Frost, E. M. Rathje, and F. E. Garcia. 2019. “Recent advances in geotechnical postearthquake reconnaissance.” Front. Built Environ. 5 (Jan): 5. https://www.frontiersin.org/articles/10.3389/fbuil.2019.00005/full.
Chattopadhay, A., A. Sarkar, P. Howlader, and V. N. Balasubramanian. 2018. “Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks.” In Proc., 2018 IEEE Winter Conf. on Applications of Computer Vision (WACV), 839–847. New York: IEEE. https://ieeexplore.ieee.org/document/8354201.
Datacenterhub. 2014. “DEEDS.” Accessed September 1, 2021. https://datacenterhub.org.
DesignSafe-CI. 2016. “Rapid experimental facility.” Accessed September 1, 2021. https://rapid.designsafe-ci.org.
EERI (Earthquake Engineering Research Institute). 2016. Learning from earthquakes.
Gao, Y., and K. M. Mosalam. 2020. “PEER Hub ImageNet: A large-scale multiattribute benchmark data set of structural images.” J. Struct. Eng. 146 (10): 04020198. https://doi.org/10.1061/(ASCE)ST.1943-541X.0002745.
GNDR (Global Network of Civil Society Organizations for Disaster Reduction). 2019. Challenge fund: Program overview (English). Washington, DC: World Bank Group.
Jafarzadeh, R., J. M. Ingham, and S. Wilkinson. 2015. “NEES: A database for seismic retrofit construction cost of concrete and steel framed schools in Iran.” Accessed September 1, 2021. https://datacenterhub.org/resources/252.
Laughery, L. A., A. Y. Puranam, C. L. Segura Jr., and A. A. Behrouzi. 2020. “The institute’s team for damage investigations.” Concr. Int. 42 (12): 32–40. https://www.concrete.org/publications/internationalconcreteabstractsportal.aspx?m=details&ID=51730377.
NIST (National Institute of Standards and Technology). 2016. “The disaster and failure studies program.” Accessed September 1, 2021. https://www.nist.gov/disaster-failure-studies/about-disaster-failure-studies-program.
Purdue University and NCREE (National Center for Research on Earthquake Engineering). 2016. “Performance of reinforced concrete buildings in the 2016 Taiwan (Meinong) earthquake.” Accessed September 1, 2021. https://datacenterhub.org/resources/14098.
SAC Joint Venture. 2000. Recommended postearthquake evaluation and repair criteria for welded steel moment-frame buildings. Washington, DC: Applied Technology Council.
Shah, P., S. Pujol, A. Puranam, and L. Laughery. 2015. “Database on performance of low-rise reinforced concrete buildings in the 2015 Nepal earthquake.” Accessed September 1, 2021. https://datacenterhub.org/resources/238.
Sim, C., C. Song, N. Skok, A. Irfanoglu, S. Pujol, and M. Sozen. 2015. “Database of low-rise reinforced concrete buildings with earthquake damage.” Accessed September 1, 2021. https://datacenterhub.org/resources/123.
Sim, C., E. Villalobos, J. P. Smith, P. Rojas, S. Pujol, A. Y. Puranam, and L. A. Laughery. 2017. Performance of low-rise reinforced concrete buildings in the 2016 Ecuador earthquake. Lafayette, IN: Purdue Univ.
StEER Network. 2018. “Structural extreme events reconnaissance network.” Accessed September 1, 2021. https://www.steer.network.
Tan, M., and Q. V. Le. 2019. “EfficientNet: Rethinking model scaling for convolutional neural networks.” Preprint, submitted May 28, 2019. https://arxiv.org/abs/1905.11946.
Yeum, C. M., S. J. Dyke, B. Benes, T. Hacker, J. Ramirez, A. Lund, and S. Pujol. 2019. “Postevent reconnaissance image documentation using automated classification.” J. Perform. Constr. Facil. 33 (1): 04018103. https://doi.org/10.1061/(ASCE)CF.1943-5509.0001253.
Yeum, C. M., S. J. Dyke, and J. Ramirez. 2018. “Visual data classification in post-event building reconnaissance.” Eng. Struct. 155 (Jan): 16–24. https://doi.org/10.1016/j.engstruct.2017.10.057.
Information & Authors
Information
Published In
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Feb 4, 2022
Accepted: May 16, 2022
Published online: Oct 12, 2022
Published in print: Dec 1, 2022
Discussion open until: Mar 12, 2023
ASCE Technical Topics:
- Artificial intelligence and machine learning
- Business management
- Comparative studies
- Computer programming
- Computing in civil engineering
- Construction engineering
- Construction management
- Design (by type)
- Earthquakes
- Engineering fundamentals
- Geohazards
- Geotechnical engineering
- Methodology (by type)
- Neural networks
- Organizations
- Personnel management
- Practice and Profession
- Research methods (by type)
- Standards and codes
- Structural design
- Structural reliability
- Team building
- Validation
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.