Abstract
Damage assessment of the built infrastructure forms a critical step in post-disaster response as it is necessary for estimating the severity and extent of the disaster impact, thereby ensuring effective and adequate recovery strategies. Contrary to traditional expert-driven approaches, recent trends show growing popularity in exploring more advanced alternate solutions, such as artificial intelligence (AI) and citizen science. One major current limitation, however, is the potential lack of reliability of these approaches. While recent efforts in the disaster research domain have successfully developed and demonstrated the use of AI and crowdsourcing-based solutions for large-scale post-disaster damage assessment, the inherent uncertainty associated with the adoption of such techniques for complicated subjective and expert-reliant tasks still hampers their practical implementation. This study aims to address this issue by reducing the uncertainty and increasing the consistency in post-disaster damage assessment by developing a novel crowd-AI framework that leverages the collective power of AI with citizen science. The framework comprises two modules: (1) an uncertainty-aware AI-assisted building damage classification module; and (2) a crowd-based probabilistic module for participatory damage assessment. Mainly, the framework uses AI predictions and the underlying uncertainty as prior knowledge in a Bayesian setting to achieve an enhanced crowd-based damage assessment. This paper presents a case study and validates that this innovative crowd-AI approach can reduce the uncertainty by as much as 83%, depending on the end-user’s uncertainty tolerance setting.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
The authors would like to thank Texas A&M University’s High Performance Research Computing for providing necessary computing infrastructure for model training. Any opinions, findings, conclusions, and recommendations expressed in this paper are those of the authors and do not necessarily represent the views of the HPRC.
References
Akter, S., and S. F. Wamba. 2019. “Big data and disaster management: A systematic review and agenda for future research.” Ann. Oper. Res. 283 (1): 939–959. https://doi.org/10.1007/s10479-017-2584-2.
Baecher, G., M. Bensi, A. Reilly, B. Phillips, L. E. Link, S. Knight, and G. Galloway. 2019. “Resiliently engineered flood and hurricane infrastructure: Principles to guide the next generation of engineers.” Bridge 49 (2): 12–16.
Barnes, E. A., R. J. Barnes, and N. Gordillo. 2021. “Adding uncertainty to neural network regression tasks in the geosciences.” Preprint, submitted September 15, 2021. https://doi.org/10.48550/arXiv.2109.07250.
Barrington, L., S. Ghosh, M. Greene, S. Har-Noy, J. Berger, S. Gill, A. Y.-M. Lin, and C. Huyck. 2012. “Crowdsourcing earthquake damage assessment using remote sensing imagery.” Annals Geophys. 54 (6): 680–687. https://doi.org/10.4401/ag-5324.
Brabham, D. C. 2008. “Crowdsourcing as a model for problem solving: An introduction and cases.” Convergence 14 (1): 75–90. https://doi.org/10.1177/1354856507084420.
Bradley, R. A., and M. E. Terry. 1952. “Rank analysis of incomplete block designs: I. The method of paired comparisons.” Biometrika 39 (3–4): 324–345. https://doi.org/10.2307/2334029.
Cannon, A. 2011. “Quantile regression neural networks: Implementation in R and application to precipitation downscaling.” Comput. Geosci. 37 (9): 1277–1284. https://doi.org/10.1016/j.cageo.2010.07.005.
Cheng, C. S., A. H. Behzadan, and A. Noshadravan. 2021. “Deep learning for post-hurricane aerial damage assessment of buildings.” Comput.-Aided Civ. Infrastruct. Eng. 36 (6): 695–710. https://doi.org/10.1111/mice.12658.
Cheng, C.-S., A. H. Behzadan, and A. Noshadravan. 2022a. “Bayesian inference for uncertainty-aware post-disaster damage assessment using artificial intelligence.” In Computing in civil engineering, 156–163. Reston, VA: ASCE.
Cheng, C.-S., A. H. Behzadan, and A. Noshadravan. 2022b. “Exploring the generalizability of deep convolutional neural networks for post-hurricane damage assessment.” In Lifelines, 577–585. Reston, VA: ASCE.
Cheng, C.-S., A. H. Behzadan, and A. Noshadravan. 2022c. “Uncertainty-aware convolutional neural network for explainable artificial intelligence-assisted disaster damage assessment.” Struct. Control Health Monit. 29 (10): e3019. https://doi.org/10.1002/stc.3019.
Choi, J., C. M. Yeum, S. J. Dyke, and M. R. Jahanshahi. 2018. “Computer-aided approach for rapid post-event visual evaluation of a building façade.” Sensors 18 (9): 3017. https://doi.org/10.3390/s18093017.
Dell’Acqua, F., and P. Gamba. 2012. “Remote sensing and earthquake damage assessment: Experiences, limits, and perspectives.” Proc. IEEE 100 (10): 2876–2890. https://doi.org/10.1109/JPROC.2012.2196404.
FEMA. 2003. Multi-hazard loss estimation methodology: Hurricane model HAZUS-MH MR3, Technical manual. Washington, DC: FEMA.
Gal, Y., and Z. Ghahramani. 2016. “Dropout as a Bayesian approximation: Representing model uncertainty in deep learning.” In Proc., 33rd Int. Conf. on Machine Learning, 1050–1059. Cambridge, MA: Proceedings of Machine Learning Research.
Gharaibeh, N., I. Oti, M. Meyer, M. Hendricks, and S. J. R. A. Van Zandt. 2021. “Potential of citizen science for enhancing infrastructure monitoring data and decision-support models for local communities.” Risk Anal. 41 (7): 1104–1110. https://doi.org/10.1111/risa.13256.
Ghosh Mondal, T., M. R. Jahanshahi, R.-T. Wu, and Z. Y. Wu. 2020. “Deep learning-based multi-class damage detection for autonomous post-disaster reconnaissance.” Struct. Control Health Monit. 27 (4): e2507. https://doi.org/10.1002/stc.2507.
Graves, A. 2011. “Practical variational inference for neural networks.” In Proc., Advances in Neural Information Processing Systems, 2348–2356. Princeton, NJ: Citeseer.
Gupta, R., R. Hosfelt, S. Sajeev, N. Patel, B. Goodman, J. Doshi, E. Heim, H. Choset, and M. Gaston. 2019. “xBD: A dataset for assessing building damage from satellite imagery.” Preprint, submitted November 21, 2019. https://doi.org/10.48550/arXiv.1911.09296.
Hendricks, M. D., M. A. Meyer, and S. M. Wilson. 2022. “Moving up the ladder in rising waters: Community science in infrastructure and hazard mitigation planning as a pathway to community control and flood disaster resilience.” Citiz. Sci.: Theory Pract. 7 (1): 18. https://doi.org/10.5334/cstp.462.
Hersbach, H. 2000. “Decomposition of the continuous ranked probability score for ensemble prediction systems.” Weather Forecasting 15 (5): 559–570. https://doi.org/10.1175/1520-0434(2000)015%3C0559:DOTCRP%3E2.0.CO;2.
Hou, L., C.-P. Yu, and D. Samaras. 2016. “Squared earth mover’s distance-based loss for training deep neural networks.” Preprint, submitted November 17, 2016. https://doi.org/10.48550/arXiv.1611.05916.
Howard, A. G., M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam. 2017. “Mobilenets: Efficient convolutional neural networks for mobile vision applications.” Preprint, submitted April 17, 2017. https://doi.org/10.48550/arXiv.1704.04861.
Hsueh, P.-Y., P. Melville, and V. Sindhwani. 2009. “Data quality from crowdsourcing: A study of annotation selection criteria.” In Proc., NAACL HLT 2009 Workshop on Active Learning for Natural Language Processing, 27–35. Stroudsburg, PA: Association for Computational Linguistics.
Jeffreys, H. 1946. “An invariant form for the prior probability in estimation problems.” Proc. R. Soc. London, Ser. A: Math. Phys. Sci. 186 (1007): 453–461. https://doi.org/10.1098/rspa.1946.0056.
Kamar, E., S. Hacker, and E. Horvitz. 2012. “Combining human and machine intelligence in large-scale crowdsourcing.” Proc. AAMAS, 467–474. Pittsburgh: International Foundation for Autonomous Agents and Multiagent Systems.
Kankanamge, N., T. Yigitcanlar, A. Goonetilleke, and M. J. I. Kamruzzaman. 2019. “Can volunteer crowdsourcing reduce disaster risk? A systematic review of the literature.” Syst. Rev. Lit. 35 (Apr): 101097. https://doi.org/10.1016/j.ijdrr.2019.101097.
Kaur, N., C.-C. Lee, A. Mostafavi, and A. Mahdavi-Amiri. 2022. “DAHiTrA: Damage assessment using a novel hierarchical transformer architecture.” Preprint, submitted April 17, 2017. https://doi.org/10.48550/arXiv.2208.02205.
Kendall, A., and Y. Gal. 2017. “What uncertainties do we need in Bayesian deep learning for computer vision?” In Proc., Advances in Neural Information Processing Systems, 5574–5584. Red Hook, NY: Curran Associates.
Khajwal, A. B., C. S. Cheng, and A. Noshadravan. 2022a. “Post-disaster damage classification based on deep multi-view image fusion.” Comput.-Aided Civ. Infrastruct. Eng. 38 (4): 528–544. https://doi.org/10.1111/mice.12890.
Khajwal, A. B., and A. Noshadravan. 2021. “An uncertainty-aware framework for reliable disaster damage assessment via crowdsourcing.” Int. J. Disaster Risk Reduct. 55 (Mar): 102110. https://doi.org/10.1016/j.ijdrr.2021.102110.
Khajwal, A. B., L. Tomotaki, C.-S. Cheng, and A. Noshadravan. 2022b. MV-HarveyNET: A labelled image dataset from Hurricane Harvey for damage assessment of residential houses based on multi-view CNN. Corvallis, OR: DesignSafe-CI. https://doi.org/10.17603/ds2-wzac-h261.
Kingma, D. P., and J. Ba. 2014. Adam: A method for stochastic optimization. London: SAGE.
Lindell, M. K., and R. W. Perry. 2003. Communicating environmental risk in multiethnic communities. Thousand Oaks, CA: SAGE.
Lindell, M. K., and C. S. Prater. 2003. “Assessing community impacts of natural disasters.” Nat. Hazards Rev. 4 (4): 176–185. https://doi.org/10.1061/(ASCE)1527-6988(2003)4:4(176).
Loos, S., et al. 2018. “Crowd-sourced remote assessments of regional-scale post-disaster damage.” In Proc., 11th US National Conf. on Earthquake Engineering. Oakland, CA: Earthquake Engineering Research Institute.
Lue, E., J. P. Wilson, and A. Curtis. 2014. “Conducting disaster damage assessments with spatial video, experts, and citizens.” Appl. Geogr. 52 (4): 46–54. https://doi.org/10.1016/j.apgeog.2014.04.014.
Negahban, S., S. Oh, and D. Shah. 2016. “Rank centrality: Ranking from pairwise com-parisons.” Oper. Res. 65 (1): 266–287. https://doi.org/10.1287/opre.2016.1534.
Nex, F., D. Duarte, F. G. Tonolo, and N. Kerle. 2019. “Structural building damage detection with deep learning: Assessment of a state-of-the-art CNN in operational conditions.” Remote Sens. 11 (23): 2765. https://doi.org/10.3390/rs11232765.
Ng, S. K., T. Krishnan, and G. J. McLachlan. 2012. “The EM algorithm.” In Handbook of computational statistics, 139–172. Berlin: Springer.
Nguyen, A., M. Halpern, B. Wallace, and M. Lease. 2016. “Probabilistic modeling for crowdsourcing partially-subjective ratings.” In Proc., Fourth AAAI Conf. on Human Computation and Crowdsourcing, 149–158. Palo Alto, CA: Association for the Advancement of Artificial Intelligence. https://doi.org/10.1609/hcomp.v4i1.13274.
Nguyen, A. T., A. Kharosekar, S. Krishnan, S. Krishnan, E. Tate, B. C. Wallace, and M. Lease. 2018. “Believe it or not: Designing a human-AI partnership for mixed-initiative fact-checking.” In Proc., 31st Annual ACM Symp. on User Interface Software and Technology, 189–199. New York: Association for Computing Machinery. https://doi.org/10.1145/3242587.3242666.
NOAA (National Oceanic and Atmospheric Administration). 2022. U.S. billion-dollar weather and climate disasters. Washington, DC: NOAA.
Paolacci, G., J. Chandler, and P. G. Ipeirotis. 2010. “Running experiments on Amazon Mechanical Turk.” Judgment Decis. Making 5 (5): 411–419. https://doi.org/10.1017/S1930297500002205.
Patel, B. N., et al. 2019. “Human-machine partnership with artificial intelligence for chest radiograph diagnosis.” NPJ Digital Med. 2: 111. https://doi.org/10.1038/s41746-019-0189-7.
Pi, Y., N. D. Nath, and A. H. Behzadan. 2020. “Convolutional neural networks for object detection in aerial imagery for disaster response and recovery.” Adv. Eng. Inf. 43 (Jan): 101009. https://doi.org/10.1016/j.aei.2019.101009.
Poblet, M., E. García-Cuesta, and P. Casanovas. 2014. “Crowdsourcing tools for disaster management: A review of platforms and methods.” In Proc., Int. Workshop on AI Approaches to the Complexity of Legal Systems, 261–274. Berlin: Springer.
Raykar, V. C., S. Yu, L. H. Zhao, G. H. Valadez, C. Florin, L. Bogoni, and L. Moy. 2010. “Learning from crowds.” J. Mach. Learn. Res. 11 (4): 1297–1322.
Roueche, D. B. L., T. Frank, I. Krupar, J. Richard, and D. J. Smith. 2018. “Collection of perishable data on wind- and surge-induced residential building damage during Hurricane Harvey (TX).” In Collection of perishable data on wind- and surge-induced residential building damage during Hurricane Harvey (TX). Corvallis, OR: DesignSafe-CI. https://doi.org/10.17603/DS2DX22.
Rudin, C., C. Chen, Z. Chen, H. Huang, L. Semenova, and C. J. S. S. Zhong. 2022. “Interpretable machine learning: Fundamental principles and 10 grand challenges.” Statistic Surv. 16 (22): 1–85. https://doi.org/10.1214/21-SS133.
Sajedi, S., and X. Liang. 2021. “Uncertainty-assisted deep vision structural health monitoring.” Comput.-Aided Civ. Infrastruct. Eng. 36 (2): 126–142. https://doi.org/10.1111/mice.12580.
Song, Z., H. Zhang, and C. Dolan. 2020. “Promoting disaster resilience: Operation mechanisms and self-organizing processes of crowdsourcing.” Sustainability 12 (5): 1862. https://doi.org/10.3390/su12051862.
Sowa, K., A. Przegalinska, and L. Ciechanowski. 2021. “Cobots in knowledge work: Human-AI collaboration in managerial professions.” J. Bus. Res. 125 (Mar): 135–142. https://doi.org/10.1016/j.jbusres.2020.11.038.
Su, J., Y. Bai, X. Wang, D. Lu, B. Zhao, H. Yang, E. Mas, and S. Koshimura. 2020. “Technical solution discussion for key challenges of operational convolutional neural network-based building-damage assessment from satellite imagery: Perspective from benchmark xBD dataset.” Remote Sens. 12 (22): 3808. https://doi.org/10.3390/rs12223808.
Sun, W., P. Bocchini, and B. D. Davison. 2020. “Applications of artificial intelligence for disaster management.” Nat. Hazards 103 (3): 2631–2689. https://doi.org/10.1007/s11069-020-04124-3.
Villarreal, M., and M. A. Meyer. 2020. “Women’s experiences across disasters: A study of two towns in Texas, United States.” Disasters 44 (2): 285–306. https://doi.org/10.1111/disa.12375.
Whitehill, J., P. L. Ruvolo, T.-F. Wu, J. Bergsma, and J. R. Movellan. 2009. “Whose vote should count more: Optimal integration of labels from labelers of unknown expertise.” In Proc., Advances in Neural Information Processing Systems, 2035–2043. Red Hook, NY: Curran Associates.
Xie, S., J. Duan, S. Liu, Q. Dai, W. Liu, Y. Ma, R. Guo, and C. Ma. 2016. “Crowdsourcing rapid assessment of collapsed buildings early after the earthquake based on aerial remote sensing image: A case study of Yushu Earthquake.” Remote Sens. 8 (9): 759. https://doi.org/10.3390/rs8090759.
Xu, J. Z., W. Lu, Z. Li, P. Khaitan, and V. Zaytseva. 2019. “Building damage detection in satellite imagery using convolutional neural networks.” Preprint, submitted October 14, 2019. https://doi.org/10.48550/arXiv.1910.06444.
Yeum, C. M., S. J. Dyke, and J. Ramirez. 2018. “Visual data classification in post-event building reconnaissance.” Eng. Struct. 155 (Jan): 16–24. https://doi.org/10.1016/j.engstruct.2017.10.057.
Yu, M., C. Yang, and Y. Li. 2018. “Big data in natural disaster management: A review.” Geosciences 8 (5): 165. https://doi.org/10.3390/geosciences8050165.
Yu, Y., X. Han, M. Yang, and J. Yang. 2020. “Probabilistic prediction of regional wind power based on spatiotemporal quantile regression.” IEEE Trans. Ind. Appl. 56 (6): 6117–6127. https://doi.org/10.1109/TIA.2020.2992945.
Yuan, F., and R. Liu. 2018. “Crowdsourcing for forensic disaster investigations: Hurricane Harvey case study.” Nat. Hazards 93 (3): 1529–1546. https://doi.org/10.1007/s11069-018-3366-0.
Zhang, Y., R. Zong, Z. Kou, L. Shang, and D. Wang. 2022. “Collablearn: An uncertainty-aware crowd-AI collaboration system for cultural heritage damage assessment.” IEEE Trans. Comput. Social Syst. 9 (5): 1515–1529. https://doi.org/10.1109/TCSS.2021.3109143.
Zhuang, Y.-T., F. Wu, C. Chen, and Y.-H. Pan. 2017. “Challenges and opportunities: From big data to knowledge in AI 2.0.” Front. Inf. Technol. Electron. Eng. 18 (1): 3–14. https://doi.org/10.1631/FITEE.1601883.
Information & Authors
Information
Published In
Journal of Engineering Mechanics
Volume 149 • Issue 9 • September 2023
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Oct 1, 2022
Accepted: Mar 28, 2023
Published online: Jun 24, 2023
Published in print: Sep 1, 2023
Discussion open until: Nov 24, 2023
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.