Deep Learning–Based Analytics of Multisource Heterogeneous Bridge Data for Enhanced Data-Driven Bridge Deterioration Prediction
Publication: Journal of Computing in Civil Engineering
Volume 36, Issue 5
Abstract
Existing data-driven bridge deterioration prediction methods mostly learn from abstract inventory data from a single source to predict the future conditions of bridges. Bridge inventory data [e.g., the National Bridge Inventory (NBI) data] are undoubtedly important but are not enough. They mainly describe bridge conditions using abstract, aggregated condition ratings that do not contain detailed information about bridge deficiencies and maintenance actions, thus limiting the performance of data-driven deterioration prediction and its usefulness in supporting bridge maintenance decision making. Learning from the wealth of heterogeneous (i.e., structured and unstructured) bridge data from multiple sources opens an unprecedented opportunity for enhanced data-driven bridge deterioration prediction. Such data include structured NBI and National Bridge Elements (NBE) data, structured traffic and weather data, and unstructured textual bridge inspection reports. To capitalize on this opportunity, this paper proposes a novel bridge data analytics framework, which allows for the extraction, integration, and analysis of structured and unstructured bridge data from different sources. At the cornerstone of this framework is a proposed deep learning–based bridge deterioration prediction method for analyzing and learning from the integrated bridge data to predict bridge deterioration. The proposed method includes three primary components: manifold learning for embedding the integrated bridge data into a low-dimensional dense space, cost-sensitive learning for modulating the misclassification costs to address the class imbalance in the data, and recurrent neural networks for learning from the embedded and balanced data from past years to predict the conditions of the primary bridge components (decks, superstructures, and substructures) in the next year. The method was evaluated in predicting the condition ratings of the decks, superstructures, and substructures of 2,646 bridges in the state of Washington. It achieved an average macroprecision and macrorecall of 89.9% and 85.8%, which are 15.0% and 22.4% higher than those achieved by learning from only NBI data.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code generated or used during the study are available in a repository or online in accordance with funder data retention policies: the NBI data [https://www.fhwa.dot.gov/bridge/nbi/ascii.cfm; FHWA (2021c)], the weather data [https://www.ncdc.noaa.gov/cdo-web/; NOAA (2021)], and the traffic data [https://www.wsdot.wa.gov/mapsdata/tools/trafficplanningtrends.htm; WSDOT (2021)], as per Table 1. Some or all data, models, or code used during the study were provided by a third party: the bridge inspection reports from the Washington State Department of Transportation, as per Table 1. Direct requests for these materials may be made to the provider as indicated in the acknowledgments. Some or all data, models, or code generated or used during the study are available from the corresponding author by request: the Python code developed for the implementation and the experimental testing of the proposed bridge deterioration prediction method.
Acknowledgments
The authors would like to thank the National Science Foundation (NSF). This paper is based upon work supported by NSF under Grant No. 1937115. Funding for this research was also provided in part by the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign through the NCSA Faculty Fellows program. This work used the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation grant number ACI-1548562. The authors would also like to thank the Washington State Department of Transportation for providing access to the bridge inspection reports. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the aforementioned entities.
References
Abdelkader, E. M., T. Zayed, and M. Marzouk. 2019. “Modelling the deterioration of bridge decks based on semi-Markov decision process.” Int. J. Strategic Decis. Sci. 10 (1): 23–45. https://doi.org/10.4018/IJSDS.2019010103.
Aremu, O. O., D. Hyland-Wood, and P. R. McAree. 2020. “A machine learning approach to circumventing the curse of dimensionality in discontinuous time series machine data.” Reliab. Eng. Syst. Saf. 195 (Mar): 106706. https://doi.org/10.1016/j.ress.2019.106706.
ASCE. 2021a. 2021 Report card for America’s infrastructure. Washington, DC: ASCE.
ASCE. 2021b. Failure to act economic reports. Washington, DC: ASCE.
Assaad, R., and I. H. El-Adaway. 2020. “Bridge infrastructure asset management system: Comparative computational machine learning approach for evaluating and predicting deck deterioration conditions.” J. Infrastruct. Syst. 26 (3): 04020032. https://doi.org/10.1061/(ASCE)IS.1943-555X.0000572.
Chang, M., M. Maguire, and Y. Sun. 2018. “Stochastic modeling of bridge deterioration using classification tree and logistic regression.” J. Infrastruct. Syst. 25 (1): 04018041. https://doi.org/10.1061/(ASCE)IS.1943-555X.0000466.
Che, C., C. Xiao, J. Liang, B. Jin, J. Zho, and F. Wang. 2017. An RNN architecture with dynamic temporal matching for personalized predictions of Parkinson’s disease.” In Proc., 2017 SIAM Int. Conf. Data Mining, 198–206. Philadelphia: Society for Industrial and Applied Mathematics.
Cho, K., B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio. 2014. “Learning phrase representations using RNN encoder-decoder for statistical machine translation.” Preprint, submitted June 6, 2014. http://arxiv.org/abs/1406.1078.
Chollet, F. 2015. “Keras.” Accessed June 21, 2021. https://github.com/fchollet/keras.
Contreras-Nieto, C., P. Lewis, and Y. Shan. 2016. “Developing predictive models of superstructure ratings for steel and prestressed concrete bridges.” In Proc., 2016 Construction Research Congress, 859–868. Reston, VA: ASCE.
Creary, P. A., and F. C. Fang. 2014. “Forecasting long-term bridge deterioration conditions using artificial intelligence techniques.” Int. J. Intel. Syst. Tech. Appl. 13 (4): 280–293. https://doi.org/10.1504/IJISTA.2014.068830.
De Boer, P. T., D. P. Kroese, S. Mannor, and R. Y. Rubinstein. 2005. “A tutorial on the cross-entropy method.” Ann. Oper. Res. 134 (1): 19–67. https://doi.org/10.1007/s10479-005-5724-z.
Dong, Q., S. Gong, and X. Zhu. 2018. “Imbalanced deep learning by minority class incremental rectification.” IEEE Trans. Pattern Anal. Mach. Intel. 41 (6): 1367–1381. https://doi.org/10.1109/TPAMI.2018.2832629.
Fan, M., H. Qiao, B. Zhang, and X. Zhang. 2012. “Isometric multi-manifold learning for feature extraction.” In Proc., 2012 IEEE 12th Int. Conf. on Data Mining, 241–250. Piscataway, NJ: IEEE.
Fang, Y., and L. Sun. 2018. “A Weibull distribution based semi-Markov process model for urban bridge deterioration prediction.” In Proc., Annual Meeting of Transportation Research Board. Washington, DC: Transportation Research Board.
FHWA (Federal Highway Administration). 1995. “Recording and coding guide for the structure inventory and appraisal of the nation’s bridges.” Accessed October 13, 2021. https://www.fhwa.dot.gov/bridge/mtguide.pdf.
FHWA (Federal Highway Administration). 2018. Bridge preservation guide: Maintaining a resilient infrastructure to preserve mobility. Washington, DC: FHWA.
FHWA (Federal Highway Administration). 2021a. “Bridge deck condition rating forecast—Machine learning models.” Accessed June 4, 2021. https://highways.dot.gov/research/long-term-infrastructure-performance/ltbp/data-analysis/bridge-deck-condition-rating-machine.
FHWA (Federal Highway Administration). 2021b. “Long-term bridge performance.” Accessed June 4, 2021. https://highways.dot.gov/research/long-term-infrastructure-performance/ltbp/long-term-bridge-performance.
FHWA (Federal Highway Administration). 2021c. “National bridge inventory.” Accessed June 21, 2021. https://www.fhwa.dot.gov/bridge/nbi/ascii.cfm.
Goyal, R., M. J. Whelan, and T. L. Cavalline. 2017. “Characterising the effect of external factors on deterioration rates of bridge components using multivariate proportional hazards regression.” Struct. Infrastruct. Eng. 13 (7): 894–905. https://doi.org/10.1080/15732479.2016.1217888.
Graves, A., A. R. Mohamed, and G. Hinton. 2013. “Speech recognition with deep recurrent neural networks.” In Proc., 2013 IEEE Int. Conf. on Acoustics, Speech and Signal, 6645–6649. Piscataway, NJ: IEEE.
Hatami, A., and G. Morcous. 2011. Developing deterioration models for Nebraska bridges. Lincoln, NE: Nebraska Department of Roads.
Hochreiter, S., and J. Schmidhuber. 1997. “Long short-term memory.” Neural Comput. 9 (8): 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735.
Huang, Y. H. 2010. “Artificial neural network model of bridge deterioration.” J. Perform. Constr. Facil. 24 (6): 597–602. https://doi.org/10.1061/(ASCE)CF.1943-5509.0000124.
Leevy, J. L., T. M. Khoshgoftaar, R. A. Bauder, and N. Seliya. 2018. “A survey on addressing high-class imbalance in big data.” J. Big Data 5 (1): 1–30. https://doi.org/10.1186/s40537-018-0151-6.
Lim, S., and S. Chi. 2019. “Xgboost application on bridge management systems for proactive damage estimation.” Adv. Eng. Inf. 41 (Aug): 100922. https://doi.org/10.1016/j.aei.2019.100922.
Lin, T. Y., P. Goyal, R. Girshick, K. He, and P. Dollár. 2017. “Focal loss for dense object detection.” In Proc., IEEE Int. Conf. on Computer Vision, 2980–2988. Piscataway, NJ: IEEE.
Liu, H., and Y. Zhang. 2020. “Bridge condition rating data modeling using deep learning algorithm.” Struct. Infrastruct. Eng. 16 (10): 1447–1460. https://doi.org/10.1080/15732479.2020.1712610.
Liu, K., and N. El-Gohary. 2017. “Ontology-based semi-supervised conditional random fields for automated information extraction from bridge inspection reports.” Autom. Constr. 81 (Sep): 313–327. https://doi.org/10.1016/j.autcon.2017.02.003.
Liu, K., and N. El-Gohary. 2020. “A smart bridge data analytics framework for enhanced bridge deterioration prediction.” In Proc., Construction Research Congress 2020, 1194–1202. Reston, VA: ASCE.
Liu, K., and N. El-Gohary. 2021a. “Fusing data extracted from bridge inspection reports for enhanced data-driven bridge deterioration prediction: A hybrid data fusion method.” J. Comput. Civ. Eng. 34 (6): 04020047. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000921.
Liu, K., and N. El-Gohary. 2021b. “Semantic neural network ensemble for automated dependency relation extraction from bridge inspection reports.” J. Comput. Civ. Eng. 35 (4): 04021007. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000961.
Liu, K., and N. El-Gohary. 2022. “Improved similarity assessment and spectral clustering for unsupervised linking of data extracted from bridge inspection reports.” Adv. Eng. Inf. 51 (Jan): 101496. https://doi.org/10.1016/j.aei.2021.101496.
Lu, P., H. Wang, and D. Tolliver. 2019. “Prediction of bridge component ratings using ordinal logistic regression model.” Math. Probl. Eng. 2019: 1–11. https://doi.org/10.1155/2019/9797584.
Madjarov, G., D. Kocev, D. Gjorgjevikj, and S. Džeroski. 2012. “An extensive experimental comparison of methods for multi-label learning.” Pattern Recognit. 45 (9): 3084–3104. https://doi.org/10.1016/j.patcog.2012.03.004.
Morcous, G. 2006. “Performance prediction of bridge deck systems using Markov chains.” J Perform. Constr. Facil. 20 (2): 146–155. https://doi.org/10.1061/(ASCE)0887-3828(2006)20:2(146).
Morcous, G., H. Rivard, and A. M. Hanna. 2002. “Modeling bridge deterioration using case-based reasoning.” J. Infrastruct. Syst. 8 (3): 86–95. https://doi.org/10.1061/(ASCE)1076-0342(2002)8:3(86).
NASEM (National Academies of Sciences, Engineering, and Medicine). 2015. Long-term bridge performance committee letter report: February 23, 2016. Washington, DC: National Academies Press.
NOAA (National Oceanic and Atmospheric Administration). 2021. “Climate data online.” Accessed June 14, 2021. https://www.ncdc.noaa.gov/cdo-web/.
Pedregosa, F., et al. 2011. “Scikit-learn: Machine learning in Python.” J. Mach. Learn. Res. 12: 2825–2830.
Qiao, Y., M. Moomen, Z. Zhang, B. Agbelie, S. Labi, and K. C. Sinha. 2016. “Modeling deterioration of bridge components with binary probit techniques with random effects.” Trans. Res. Rec. 2550 (1): 96–105. https://doi.org/10.3141/2550-13.
Samko, O., A. D. Marshall, and P. L. Rosin. 2006. “Selection of the optimal parameter value for the Isomap algorithm.” Pattern Recognit. Lett. 27 (9): 968–979. https://doi.org/10.1016/j.patrec.2005.11.017.
Tang, J., F. Belletti, S. Jain, M. Chen, A. Beutel, C. Xu, and H. E. Chi. 2019. “Towards neural mixture recommender for long range dependent user sequences.” In Proc., World Wide Web Conf., 1782–1793. Geneva: International World Wide Web Conference Committee.
Tenenbaum, J. B., V. De Silva, and J. C. Langford. 2000. “A global geometric framework for nonlinear dimensionality reduction.” Science 290 (5500): 2319–2323. https://doi.org/10.1126/science.290.5500.2319.
Varshney, K. R., and A. S. Willsky. 2011. “Linear dimensionality reduction for margin-based classification: High-dimensional data and sensor networks.” IEEE Trans. Signal Process. 59 (6): 2496–2512. https://doi.org/10.1109/TSP.2011.2123891.
Wellalage, N. K. W., T. Zhang, and R. Dwight. 2014. “Calibrating Markov chain–based deterioration models for predicting future conditions of railway bridge elements.” J. Bridge Eng. 20 (2): 04014060. https://doi.org/10.1061/(ASCE)BE.1943-5592.0000640.
WSDOT (Washington DOT). 2015. 2015 Bridge inspection report for bridge 0005115A. Olympia, WA: WSDOT.
WSDOT (Washington DOT). 2016. 2016 Bridge inspection report for bridge 0000965A. Olympia, WA: WSDOT.
WSDOT (Washington DOT). 2021. “Traffic data GeoPortal.” Accessed June 14, 2021. https://www.wsdot.wa.gov/mapsdata/tools/trafficplanningtrends.htm.
Zambon, I., A. Vidovic, A. Strauss, J. Matos, and J. Amado. 2017. “Comparison of stochastic prediction models based on visual inspections of bridge decks.” J. Civ. Eng. Manage. 23 (5): 553–561. https://doi.org/10.3846/13923730.2017.1323795.
Zhang, H., and D. W. R. Marsh. 2020. “Multi-state deterioration prediction for infrastructure asset: Learning from uncertain data, knowledge and similar groups.” Inf. Sci. 529 (Aug): 197–213. https://doi.org/10.1016/j.ins.2019.11.017.
Information & Authors
Information
Published In
Journal of Computing in Civil Engineering
Volume 36 • Issue 5 • September 2022
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Aug 8, 2021
Accepted: Dec 15, 2021
Published online: Jun 30, 2022
Published in print: Sep 1, 2022
Discussion open until: Nov 30, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Mahdi Ghafoori, Moatassem Abdallah, Mehmet Egemen Ozbek, Modeling and Predicting Deterioration of Concrete Bridge Elements Using Machine Learning, Construction Research Congress 2024, 10.1061/9780784485279.077, (769-777), (2024).
- Qiyang Chen, Nora El-Gohary, Deep Learning-Based Coreference Resolution for Bridge Report Analytics, Construction Research Congress 2024, 10.1061/9780784485262.026, (249-258), (2024).
- Xi Hu, Kaijian Liu, Structural Deterioration Knowledge Ontology towards Physics-Informed Machine Learning for Enhanced Bridge Deterioration Prediction, Journal of Computing in Civil Engineering, 10.1061/(ASCE)CP.1943-5487.0001066, 37, 1, (2023).