Model Falsification from a Bayesian Viewpoint with Applications to Parameter Inference and Model Selection of Dynamical Systems
Publication: Journal of Engineering Mechanics
Volume 150, Issue 6
Abstract
This work provides a Bayesian reinterpretation of model falsification. We show that model falsification can be viewed as an approximate Bayesian computation (ABC) approach when hypotheses (models) are sampled from a prior. To achieve this, we recast model falsifiers as discrepancy metrics and density kernels such that they may be adopted within ABC and generalized ABC (GABC) methods. We call the resulting frameworks model-falsified ABC and model-falsified GABC, respectively. As a result of our reinterpretation, the set of unfalsified models can be shown to be realizations of an approximate posterior. We consider both error- and likelihood-domain model falsification. Model-falsified (G)ABC is used to address two practical types of inverse problems, although with synthetic measurements. The first two problems concern parameter estimation and include applications of ABC to the inference of a statistical model in which the likelihood can be difficult to compute, and the identification of a cubic-quintic dynamical system. The third example involves model selection for the base isolation system of a four-degree-of-freedom base-isolated structure. The performance of model-falsified ABC and GABC is compared with that of Bayesian inference. The results show that model-falsified (G)ABC can be used to solve inverse problems in a computationally efficient manner. The results are also used to compare the various falsifiers in terms of their capability to approximate the posterior and some of its important statistics. Furthermore, we show that model falsifier–based density kernels can be used in kernel regression to infer unknown model parameters and compute structural responses under epistemic uncertainty.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
The data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request, with the exception of the El Centro and Ridgecrest earthquake records, which can be obtained from Center for Engineering Strong Motion Data (2019) and Strong Motion Virtual Data Center (2019), and the open source pyABC toolbox (Klinger et al. 2018).
Acknowledgments
The authors gratefully acknowledge the support of this work by the National Science Foundation (NSF) through Award CMMI 16-63667. The first author also acknowledges support from the University of Southern California (USC) through a Provost’s Ph.D. Fellowship. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of NSF or USC. The authors also acknowledge the Center for Advanced Research Computing (CARC) at the University of Southern California for providing computing resources that contributed to the research results reported within this publication. The authors further acknowledge accessing strong-motion data through the Center for Engineering Strong Motion Data (CESMD), last visited on November 17, 2021; the networks or agencies providing the data used in this report are the California Strong Motion Instrumentation Program (CSMIP) and the USGS National Strong Motion Project (NSMP).
References
Abdessalem, A. B., N. Dervilis, D. Wagg, and K. Worden. 2018. “Model selection and parameter estimation in structural dynamics using approximate Bayesian computation.” Mech. Syst. Signal Process. 99 (Jan): 306–325. https://doi.org/10.1016/j.ymssp.2017.06.017.
Abdessalem, A. B., N. Dervilis, D. Wagg, and K. Worden. 2019. “Model selection and parameter estimation of dynamical systems using a novel variant of approximate Bayesian computation.” Mech. Syst. Signal Process. 122 (May): 364–386. https://doi.org/10.1016/j.ymssp.2018.12.048.
Abdi, H. 2007. “Bonferroni and Šidák corrections for multiple comparisons.” In Vol. 3 of Encyclopedia of measurement and statistics, 103–107. Thousand Oaks, CA: SAGE.
Barber, S., J. Voss, and M. Webster. 2015. “The rate of convergence for approximate Bayesian computation.” Electron. J. Stat. 9 (1): 80–105. https://doi.org/10.1214/15-EJS988.
Barros, J., M. Chiachío, J. Chiachío, and F. Cabanilla. 2022. “Adaptive approximate Bayesian computation by subset simulation for structural model calibration.” Comput.-Aided Civ. Infrastruct. Eng. 37 (6): 726–745. https://doi.org/10.1111/mice.12762.
Beaumont, M. A., J.-M. Cornuet, J.-M. Marin, and C. P. Robert. 2009. “Adaptive approximate Bayesian computation.” Biometrika 96 (4): 983–990. https://doi.org/10.1093/biomet/asp052.
Beaumont, M. A., W. Zhang, and D. J. Balding. 2002. “Approximate Bayesian computation in population genetics.” Genetics 162 (4): 2025–2035. https://doi.org/10.1093/genetics/162.4.2025.
Benjamini, Y., and Y. Hochberg. 1995. “Controlling the false discovery rate: A practical and powerful approach to multiple testing.” J. R. Stat. Soc. B 57 (1): 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x.
Beven, K. J. 1993. “Prophecy, reality and uncertainty in distributed hydrological modelling.” Adv. Water Resour. 16 (1): 41–51. https://doi.org/10.1016/0309-1708(93)90028-E.
Beven, K. J. 2011. Rainfall-runoff modelling: The primer. New York: Wiley.
Beven, K. J., and J. Freer. 2001. “Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems using the GLUE methodology.” J. Hydrol. 249 (1–4): 11–29. https://doi.org/10.1016/S0022-1694(01)00421-8.
Blum, M. G. B. 2010. “Approximate Bayesian computation: A nonparametric perspective.” J. Am. Stat. Assoc. 105 (491): 1178–1187. https://doi.org/10.1198/jasa.2010.tm09448.
Center for Engineering Strong Motion Data. 2019. “Information for strong-motion station tower 2: SCSN station TOW2.” Accessed November 17, 2021. https://www.strongmotioncenter.org/cgi-bin/CESMD/stationhtml.pl?stationID=CITOW2&network=SCSN.
Chiachio, M., J. L. Beck, J. Chiachio, and G. Rus. 2014. “Approximate Bayesian computation by subset simulation.” SIAM J. Sci. Comput. 36 (3): A1339–A1358. https://doi.org/10.1137/130932831.
Dasgupta, A. 2023. “Model falsification: New insights, methods and applications.” Ph.D. thesis, Sonny Astani Dept. of Civil and Environmental Engineering, Univ. of Southern California.
Dasgupta, A., and E. A. Johnson. 2022. “Model falsification from a Bayesian viewpoint with applications to system identification and model selection.” In Proc., 8th World Conf. on Structural Control and Monitoring (8WCSCM). Berlin: Springer.
De, S., P. T. Brewick, E. A. Johnson, and S. F. Wojtkiewicz. 2018. “Investigation of model falsification using error and likelihood bounds with application to a structural system.” J. Eng. Mech. 144 (9): 04018078. https://doi.org/10.1061/(ASCE)EM.1943-7889.0001440.
Del Moral, P., A. Doucet, and A. Jasra. 2012. “An adaptive sequential Monte Carlo method for approximate Bayesian computation.” Stat. Comput. 22 (5): 1009–1020. https://doi.org/10.1007/s11222-011-9271-y.
Durrett, R. 2019. Probability: Theory and examples. 4th ed. Cambridge, UK: Cambridge University Press.
Elías-Zúñiga, A. 2013. “Exact solution of the cubic-quintic Duffing oscillator.” Appl. Math. Modell. 37 (4): 2574–2579. https://doi.org/10.1016/j.apm.2012.04.005.
Evans, S. N., and P. B. Stark. 2002. “Inverse problems as statistics.” Inverse Probl. 18 (4): R55. https://doi.org/10.1088/0266-5611/18/4/201.
Fang, S.-E., S. Chen, Y.-Q. Lin, and Z.-L. Dong. 2019. “Probabilistic damage identification incorporating approximate Bayesian computation with stochastic response surface.” Mech. Syst. Signal Process. 128 (Aug): 229–243. https://doi.org/10.1016/j.ymssp.2019.03.044.
Fearnhead, P., and D. Prangle. 2012. “Constructing summary statistics for approximate Bayesian computation: Semi-automatic approximate Bayesian computation.” J. R. Stat. Soc. B 74 (3): 419–474. https://doi.org/10.1111/j.1467-9868.2011.01010.x.
Freedman, D. 1997. “Some issues in the foundation of statistics.” In Topics in the foundation of statistics, 19–39. New York: Springer.
Freedman, D. A., and P. B. Stark. 2003. “What is the chance of an earthquake?” In Vol. 32 of Earthquake science and seismic risk reduction (NATO Science Series: IV), edited by F. Mulargia and R. J. Geller, 201–213. Berlin: Springer.
Gelman, A., and C. R. Shalizi. 2013. “Philosophy and the practice of Bayesian statistics.” Br. J. Math. Stat. Psychol. 66 (1): 8–38. https://doi.org/10.1111/j.2044-8317.2011.02037.x.
Goulet, J.-A., S. Coutu, and I. F. Smith. 2013. “Model falsification diagnosis and sensor placement for leak detection in pressurized pipe networks.” Adv. Eng. Inf. 27 (2): 261–269. https://doi.org/10.1016/j.aei.2013.01.001.
Goulet, J.-A., P. Kripakaran, and I. F. C. Smith. 2010. “Multimodel structural performance monitoring.” J. Struct. Eng. 136 (10): 1309–1318. https://doi.org/10.1061/(ASCE)ST.1943-541X.0000232.
Goulet, J.-A., and I. F. C. Smith. 2013a. “Performance-driven measurement system design for structural identification.” J. Comput. Civ. Eng. 27 (4): 427–436. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000250.
Goulet, J.-A., and I. F. C. Smith. 2013b. “Structural identification with systematic errors and unknown uncertainty dependencies.” Comput. Struct. 128 (Nov): 251–258. https://doi.org/10.1016/j.compstruc.2013.07.009.
Hazra, I., and M. D. Pandey. 2021. “A likelihood-free approach towards Bayesian modeling of degradation growths using mixed-effects regression.” Comput. Struct. 244 (Feb): 106427. https://doi.org/10.1016/j.compstruc.2020.106427.
Hazra, I., M. D. Pandey, and N. Manzana. 2020. “Approximate Bayesian computation (ABC) method for estimating parameters of the gamma process using noisy data.” Reliab. Eng. Syst. Saf. 198 (Jun): 106780. https://doi.org/10.1016/j.ress.2019.106780.
Hwang, J., and J. Chiou. 1996. “An equivalent linear model of lead-rubber seismic isolation bearings.” Eng. Struct. 18 (7): 528–536. https://doi.org/10.1016/0141-0296(95)00132-8.
Klinger, E., D. Rickert, and J. Hasenauer. 2018. “pyABC: Distributed, likelihood-free inference.” Bioinformatics 34 (20): 3591–3593. https://doi.org/10.1093/bioinformatics/bty361.
Kroese, D. P., T. Taimre, and Z. I. Botev. 2013. Vol. 706 of Handbook of Monte Carlo methods. New York: Wiley.
Milne, P. 1995. “A Bayesian defence of Popperian science?” Analysis 55 (3): 213–215. https://doi.org/10.1093/analys/55.3.213.
Moser, G., S. G. Paal, and I. F. C. Smith. 2018. “Leak detection of water supply networks using error-domain model falsification.” J. Comput. Civ. Eng. 32 (2): 04017077. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000729.
Pai, S. G., A. Nussbaumer, and I. F. C. Smith. 2018. “Comparing structural identification methodologies for fatigue life prediction of a highway bridge.” Front. Built Environ. 3 (Jan): 73. https://doi.org/10.3389/fbuil.2017.00073.
Pai, S. G., and I. F. C. Smith. 2017. “Comparing three methodologies for system identification and prediction.” In Proc., 14th Int. Probabilistic Workshop, 81–95. New York: Springer.
Pasquier, R., and I. F. C. Smith. 2015. “Robust system identification and model predictions in the presence of systematic uncertainty.” Adv. Eng. Inf. 29 (4): 1096–1109. https://doi.org/10.1016/j.aei.2015.07.007.
Pritchard, J. K., M. T. Seielstad, A. Perez-Lezaun, and M. W. Feldman. 1999. “Population growth of human Y chromosomes: A study of Y chromosome microsatellites.” Mol. Biol. Evol. 16 (12): 1791–1798. https://doi.org/10.1093/oxfordjournals.molbev.a026091.
Ramallo, J. C., E. A. Johnson, and B. F. Spencer Jr. 2002. “‘Smart’ base isolation systems.” J. Eng. Mech. 128 (10): 1088–1099. https://doi.org/10.1061/(ASCE)0733-9399(2002)128:10(1088).
Rasmussen, C. E., and H. Nickisch. 2016. The GPML toolbox version 4.0. Cambridge, MA: MIT Press.
Ratmann, O., A. Camacho, A. Meijer, and G. Donker. 2013. “Statistical modelling of summary values leads to accurate approximate Bayesian computations.” Preprint, submitted May 18, 2013. https://arxiv.org/abs/1305.4283.
Rosenkrantz, R. D. 1977. “Bayes and Popper.” In Inference, method and decision, 118–134. New York: Springer.
Rubin, D. B. 1984. “Bayesianly justifiable and relevant frequency calculations for the applied statistician.” Ann. Stat. 12 (4): 1151–1172. https://doi.org/10.1214/aos/1176346785.
Sadegh, M., and J. Vrugt. 2013. “Bridging the gap between GLUE and formal statistical approaches: Approximate Bayesian computation.” Hydrol. Earth Syst. Sci. 17 (12): 4831–4850. https://doi.org/10.5194/hess-17-4831-2013.
Schälte, Y., and J. Hasenauer. 2020. “Efficient exact inference for dynamical systems with noisy measurements using sequential approximate Bayesian computation.” Supplement, Bioinformatics 36 (S1): i551–i559. https://doi.org/10.1093/bioinformatics/btaa397.
Sisson, S. A., Y. Fan, and M. Beaumont. 2018. Handbook of approximate Bayesian computation. Boca Raton, FL: CRC Press.
Stark, P. B., and L. Tenorio. 2010. “A primer of frequentist and Bayesian inference in inverse problems.” Chap. 2 in Large–scale inverse problems and quantification of uncertainty, 9–32. New York: Wiley.
Strong Motion Virtual Data Center. 2019. “Acceleration.” Accessed November 17, 2021. https://www.strongmotioncenter.org/vdc/scripts/plot.plx?stn=6030&evt=1350.
Tavaré, S., D. J. Balding, R. C. Griffiths, and P. Donnelly. 1997. “Inferring coalescence times from DNA sequence data.” Genetics 145 (2): 505–518. https://doi.org/10.1093/genetics/145.2.505.
Toni, T., D. Welch, N. Strelkowa, A. Ipsen, and M. P. Stumpf. 2009. “Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems.” J. R. Soc. Interface 6 (31): 187–202. https://doi.org/10.1098/rsif.2008.0172.
Vakilzadeh, M. K., Y. Huang, J. L. Beck, and T. Abrahamsson. 2017. “Approximate Bayesian computation by subset simulation using hierarchical state-space models.” Mech. Syst. Signal Process. 84 (Feb): 2–20. https://doi.org/10.1016/j.ymssp.2016.02.024.
Van Rossum, G., and F. L. Drake. 2009. Python 3 reference manual. Scotts Valley, CA: CreateSpace.
Virtanen, P., et al. 2020. “SciPy 1.0: Fundamental algorithms for scientific computing in Python.” Nat. Methods 17 (3): 261–272. https://doi.org/10.1038/s41592-019-0686-2.
Vrugt, J. A., and M. Sadegh. 2013. “Toward diagnostic model calibration and evaluation: Approximate Bayesian computation.” Water Resour. Res. 49 (7): 4335–4345. https://doi.org/10.1002/wrcr.20354.
Wasserman, L. 2006. All of nonparametric statistics. New York: Springer.
Wen, Y.-K. 1976. “Method for random vibration of hysteretic systems.” J. Eng. Mech. Div. 102 (2): 249–263. https://doi.org/10.1061/JMCEA3.0002106.
Wilkinson, R. 2014. “Accelerating ABC methods using Gaussian processes.” In Artificial intelligence and statistics, 1015–1023. Cambridge MA: Proceedings of Machine Learning Research.
Wilkinson, R. D. 2013. “Approximate Bayesian computation (ABC) gives exact results under the assumption of model error.” Stat. Appl. Genet. Mol. Biol. 12 (2): 129–141. https://doi.org/10.1515/sagmb-2013-0010.
Yuen, K.-V. 2010. Bayesian methods for structural dynamics and civil engineering. New York: Wiley.
Information & Authors
Information
Published In
Journal of Engineering Mechanics
Volume 150 • Issue 6 • June 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Feb 6, 2023
Accepted: Aug 7, 2023
Published online: Mar 18, 2024
Published in print: Jun 1, 2024
Discussion open until: Aug 18, 2024
ASCE Technical Topics:
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.