Adaptive Kriging Stochastic Sampling and Density Approximation and Its Application to Rare-Event Estimation
Publication: ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems, Part A: Civil Engineering
Volume 4, Issue 3
Abstract
This paper examines the task of approximating or generating samples according to a target probability distribution when this distribution is expressed as a function of the response of an engineering system. Frequently such approximation is performed in a sequential manner, using a series of intermediate densities that converge to the target density and may require a large number of evaluations of the system response, which for applications involving complex numerical models creates a significant computational burden. To alleviate this burden an adaptive Kriging stochastic sampling and density approximation framework (AK-SSD) is developed in this work. The metamodel approximates the system response vector, whereas the adaptive characteristics are established through an iterative approach. At the end of each iteration, the target density, approximated through the current metamodel, is compared to the density established at the previous iteration, using the Hellinger distance as a comparison metric. If convergence has not been achieved, then additional simulation experiments are performed to inform the metamodel development, through a sample-based design of experiments that balances between the improvement of the metamodel accuracy and the addition of experiments in regions of importance for the stochastic sampling. These regions are defined by considering both the target density and any intermediate densities. The process then moves to the next iteration, with an improved metamodel developed using all the available simulation experiments. Although the theoretical discussions are general, the emphasis is placed on rare-event simulation. For this application, once the target density is approximated (first stage), it is used (second stage) as an importance sampling density for estimating the rare-event likelihood. For the second stage, use of either the metamodel or the exact numerical model is examined.
Get full access to this article
View all available purchase options and get full access to this article.
References
Ang, G. L., A. H.-S. Ang, and W. H. Tang. 1992. “Optimal importance sampling density estimator.” J. Eng. Mech. 118 (6): 1146–1163. https://doi.org/10.1061/(ASCE)0733-9399(1992)118:6(1146).
Angelikopoulos, P., C. Papadimitriou, and P. Koumoutsakos. 2015. “X-TMCMC: Adaptive Kriging for Bayesian inverse modeling.” Comput. Methods Appl. Mech. Eng. 289: 409–428. https://doi.org/10.1016/j.cma.2015.01.015.
Au, S. K., and J. L. Beck. 2001. “Estimation of small failure probabilities in high dimensions by subset simulation.” Probab. Eng. Mech. 16 (4): 263–277. https://doi.org/10.1016/S0266-8920(01)00019-4.
Au, S.-K., and Y. Wang. 2014. Engineering risk assessment with subset simulation. New York: Wiley.
Balesdent, M., J. Morio, and J. Marzat. 2013. “Kriging-based adaptive importance sampling algorithms for rare event estimation.” Struct. Saf. 44: 1–10. https://doi.org/10.1016/j.strusafe.2013.04.001.
Beck, J. L. 2010. “Bayesian system identification based on probability logic.” Struct. Control Health Monit. 17 (7): 825–847. https://doi.org/10.1002/stc.424.
Beck, J. L., and A. Taflanidis. 2013. “Prior and posterior robust stochastic predictions for dynamical systems using probability logic.” J. Uncertainty Quantif. 3 (4): 271–288. https://doi.org/10.1615/Int.J.UncertaintyQuantification.2012003641.
Bect, J., L. Li, and E. Vazquez. 2016. “Bayesian subset simulation.” Preprint arXiv:1601.02557.
Behmanesh, I., B. Moaveni, G. Lombaert, and C. Papadimitriou. 2015. “Hierarchical Bayesian model updating for probabilistic damage identification.” Vol. 3 of Model validation and uncertainty quantification, 55–66. New York: Springer.
Bengtsson, T., P. Bickel, and B. Li. 2008. “Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems.” Probability and statistics: Essays in honor of David A. Freedman, 316–334. Beachwood, OH: Institute of Mathematical Statistics.
Berg, B. A. 2004. Markov Chain Monte Carlo simulations and their statistical analysis. Singapore: World Scientific.
Bichon, B. J., M. S. Eldred, L. P. Swiler, S. Mahadevan, and J. M. McFarland. 2008. “Efficient global reliability analysis for nonlinear implicit performance functions.” AIAA J. 46 (10): 2459–2468. https://doi.org/10.2514/1.34321.
Bourinet, J.-M. 2016. “Rare-event probability estimation with adaptive support vector regression surrogates.” Reliab. Eng. Syst. Saf. 150: 210–221. https://doi.org/10.1016/j.ress.2016.01.023.
Bourinet, J.-M., F. Deheeger, and M. Lemaire. 2011. “Assessing small failure probabilities by combined subset simulation and support vector machines.” Struct. Saf. 33 (6): 343–353. https://doi.org/10.1016/j.strusafe.2011.06.001.
Box, G. E. P., and G. C. Tao 1992. Bayesian inference in statistical analysis. New York: Wiley.
Cao, Y., and D. J. Fleet. 2014. “Generalized product of experts for automatic and principled fusion of Gaussian process predictions.” Preprint arXiv:1410.7827.
Ching, J., and Y.-C. Chen. 2007. “Transitional Markov chain Monte Carlo method for Bayesian model updating, model class selection, and model averaging.” J. Eng. Mech. 133 (7): 816–832. https://doi.org/10.1061/(ASCE)0733-9399(2007)133:7(816).
Chopin, N. 2002. “A sequential particle filter method for static models.” Biometrika 89 (3): 539–552. https://doi.org/10.1093/biomet/89.3.539.
Conrad, P. R., Y. M. Marzouk, N. S. Pillai, and A. Smith. 2016. “Accelerating asymptotically exact MCMC for computationally intensive models via local approximations.” J. Am. Stat. Assoc. 111 (516): 1591–1607. https://doi.org/10.1080/01621459.2015.1096787.
Dubourg, V., B. Sudret, and J.-M. Bourinet. 2011. “Reliability-based design optimization using kriging surrogates and subset simulation.” Struct. Multidiscip. Optim. 44 (5): 673–690. https://doi.org/10.1007/s00158-011-0653-8.
Dubourg, V., B. Sudret, and F. Deheeger. 2013. “Metamodel-based importance sampling for structural reliability analysis.” Probab. Eng. Mech. 33: 47–57. https://doi.org/10.1016/j.probengmech.2013.02.002.
Echard, B., N. Gayton, and M. Lemaire. 2011. “AK-MCS: an active learning reliability method combining Kriging and Monte Carlo simulation.” Struct. Saf. 33 (2): 145–154. https://doi.org/10.1016/j.strusafe.2011.01.002.
Fu, S., M. Couplet, and N. Bousquet. 2017. “An adaptive Kriging method for solving nonlinear inverse statistical problems.” Environmetrics 28 (4): e2439. https://doi.org/10.1002/env.2439.
Gidaris, I., A. A. Taflanidis, and G. P. Mavroeidis. 2015. “Kriging metamodeling in seismic risk assessment based on stochastic ground motion models.” Earthquake Eng. Struct. Dyn. 44 (14): 2377–2399. https://doi.org/10.1002/eqe.2586.
Giovanis, D. G., I. Papaioannou, D. Straub, and V. Papadopoulos. 2017. “Bayesian updating with subset simulation using artificial neural networks.” Comput. Methods Appl. Mech. Eng. 319: 124–145. https://doi.org/10.1016/j.cma.2017.02.025.
Hartigan, J. A., and M. A. Wong. 1979. “Algorithm AS 136: A K-means clustering algorithm.” J. R. Stat. Soc. Ser. C Appl. Stat. 28 (1): 100–108. https://doi.org/10.2307/2346830.
Hesterberg, T. 1995. “Weighted average importance sampling and defensive mixture distributions.” Technometrics 37 (2): 185–194. https://doi.org/10.1080/00401706.1995.10484303.
Huang, X., J. Chen, and H. Zhu. 2016. “Assessing small failure probabilities by AK-SS: An active learning method combining Kriging and subset simulation.” Struct. Saf. 59: 86–95. https://doi.org/10.1016/j.strusafe.2015.12.003.
Jensen, H., C. Esse, V. Araya, and C. Papadimitriou. 2017. “Implementation of an adaptive meta-model for Bayesian finite element model updating in time domain.” Reliab. Eng. Syst. Saf. 160: 174–190. https://doi.org/10.1016/j.ress.2016.12.005.
Jia, G., and A. A. Taflanidis. 2013. “Kriging metamodeling for approximation of high-dimensional wave and surge responses in real-time storm/hurricane risk assessment.” Comput. Methods Appl. Mech. Eng. 261–262: 24–38. https://doi.org/10.1016/j.cma.2013.03.012.
Jia, G., and A. A. Taflanidis. 2014. “Sample-based evaluation of global probabilistic sensitivity measures” Comput. Struct. 144: 103–118. https://doi.org/10.1016/j.compstruc.2014.07.019.
Jia, G., A. A. Taflanidis, and J. L. Beck. 2015. “A new adaptive rejection sampling method using kernel density approximations and its application to subset simulation.” J. Risk Uncertainty Eng. Syst. Part A: Civ. Eng. 3 (2): D4015001. https://doi.org/10.1061/AJRUA6.0000841.
Jones, D. R., M. Schonlau, and W. J. Welch. 1998. “Efficient global optimization of expensive black-box functions.” J. Global Optim. 13 (4): 455–492.
Kleijnen, J. P. C. 2008. Design and analysis of simulation experiments. New York: Springer.
Kleijnen, J. P. C. 2009. “Kriging metamodeling in simulation: A review.” Eur. J. Oper. Res. 192 (3): 707–716. https://doi.org/10.1016/j.ejor.2007.10.013.
Kroese, D. P., R. Y. Rubinstein, and P. W. Glynn. 2013. “The cross-entropy method for estimation.” Vol. 31 of Handbook of statistics: Machine learning: Theory and applications, 19–34. Amsterdam, Netherlands: Elsevier.
Kurtz, N., and J. Song. 2013. “Cross-entropy-based adaptive importance sampling using Gaussian mixture.” Struct. Saf. 42: 35–44. https://doi.org/10.1016/j.strusafe.2013.01.006.
Li, J., and Y. M. Marzouk. 2014. “Adaptive construction of surrogates for the Bayesian solution of inverse problems.” SIAM J. Sci. Comput. 36 (3): A1163–A1186. https://doi.org/10.1137/130938189.
Li, J., and D. Xiu. 2010. “Evaluation of failure probability via surrogate models.” J. Comput. Phys. 229 (23): 8966–8980. https://doi.org/10.1016/j.jcp.2010.08.022.
Lophaven, S. N., H. B. Nielsen, and J. Sondergaard. 2002. Aspects of the MATLAB toolbox DACE. Lyngby, Denmark: Technical Univ. of Denmark.
Martin, J. D., and T. W. Simpson. 2005. “Use of Kriging models to approximate deterministic computer models.” AIAA J. 43 (4): 853–863. https://doi.org/10.2514/1.8650.
Medina, J. C., and A. Taflanidis. 2014. “Adaptive importance sampling for optimization under uncertainty problems.” Comput. Methods Appl. Mech. Eng. 279: 133–162. https://doi.org/10.1016/j.cma.2014.06.025.
Moustapha, M., B. Sudret, J.-M. Bourinet, and B. Guillaume. 2016. “Quantile-based optimization under uncertainties using adaptive Kriging surrogate models.” Struct. Multi. Optim. 54 (6): 1403–1421. https://doi.org/10.1007/s00158-016-1504-4.
Neal, R. M. 2001. “Annealed importance sampling.” Stat. Comput. 11 (2): 125–139. https://doi.org/10.1023/A:1008923215028.
Papadimitriou, C., J. L. Beck, and L. S. Katafygiotis. 2001. “Updating robust reliability using structural test data.” Probab. Eng. Mech. 16 (2): 103–113. https://doi.org/10.1016/S0266-8920(00)00012-6.
Papadopoulos, V., D. G. Giovanis, N. D. Lagaros, and M. Papadrakakis. 2012. “Accelerated subset simulation with neural networks for reliability analysis.” Comput. Methods Appl. Mech. Eng. 223: 70–80. https://doi.org/10.1016/j.cma.2012.02.013.
Pedroni, N., and E. Zio. 2017. “An adaptive metamodel-based subset importance sampling approach for the assessment of the functional failure probability of a thermal-hydraulic passive system.” Appl. Math. Modell. 48: 269–288. https://doi.org/10.1016/j.apm.2017.04.003.
Picheny, V., D. Ginsbourger, O. Roustant, R. T. Haftka, and N. H. Kim. 2010. “Adaptive designs of experiments for accurate approximation of a target region.” J. Mech. Des. 132 (7): 071008. https://doi.org/10.1115/1.4001873.
Pollard, D. 2002. A user’s guide to measure theoretic probability. Cambridge, UK: Cambridge University Press.
Robert, C. P., and G. Casella. 2004. Monte Carlo statistical methods. New York: Springer.
Roberts, G. O., A. Gelman, and W. R. Gilks. 1997. “Weak convergence and optimal scaling of random walk metropolis algorithms.” Ann. Appl. Probab. 7 (1): 110–120. https://doi.org/10.1214/aoap/1034625254.
Roberts, G. O., and J. S. Rosenthal. 2004. “General state-space Markov chains and MCMC algorithms.” Probab. Surveys 1: 20–71. https://doi.org/10.1214/154957804100000024.
Rubinstein, R. Y., and D. P. Kroese. 2004. The cross-entropy method. New York: Springer.
Sacks, J., W. J. Welch, T. J. Mitchell, and H. P. Wynn. 1989. “Design and analysis of computer experiments.” Stat. Sci. 4 (4): 409–423. https://doi.org/10.1214/ss/1177012413.
Sankararaman, S., and S. Mahadevan. 2013. “Bayesian methodology for diagnosis uncertainty quantification and health monitoring.” Struct. Control Health Monit. 20 (1): 88–106. https://doi.org/10.1002/stc.476.
Schöbi, R., B. Sudret, and S. Marelli. 2016. “Rare event estimation using polynomial-chaos Kriging.” J. Risk Uncertainty Eng. Systems, Part A: Civ. Eng. 3 (2): D4016002. https://doi.org/10.1061/AJRUA6.0000870.
Song, S., Z. Lu, and H. Qiao. 2009. “Subset simulation for structural reliability sensitivity analysis.” Reliab. Eng. Syst. Saf. 94 (2): 658–665. https://doi.org/10.1016/j.ress.2008.07.006.
Spall, J. C. 2003. Introduction to stochastic search and optimization. New York: Wiley-Interscience.
Straub, D., and I. Papaioannou. 2015. “Bayesian updating with structural reliability methods.” J. Eng. Mech. 141 (3): 04014134. https://doi.org/10.1061/(ASCE)EM.1943-7889.0000839.
Sudret, B., V. Dubourg, and J. Bourinet. 2012. “Enhancing meta-model-based importance sampling by subset simulation.” In Proc., IFIP WG7.5 Working Conf. on Reliability and Optimization of Structural Systems. London: Taylor and Francis.
Tang, K., P. M. Congedo, and R. Abgrall. 2016. “Adaptive surrogate modeling by ANOVA and sparse polynomial dimensional decomposition for global sensitivity analysis in fluid simulation.” J. Comput. Phys. 314: 557–589. https://doi.org/10.1016/j.jcp.2016.03.026.
Veach, E., and L. J. Guibas. 1995. “Optimally combining sampling techniques for Monte Carlo rendering.” In Proc., 22nd Annual Conf. on Computer Graphics and Interactive Techniques, 419–428. New York: ACM.
Wang, G. G., and S. Shan. 2007. “Review of metamodeling techniques in support of engineering design optimization.” J. Mech. Des. 129 (4): 370–380. https://doi.org/10.1115/1.2429697.
Zhang, J., A. A. Taflanidis, and J. C. Medina. 2016. “Sequential approximate optimization for design under uncertainty problems utilizing Kriging metamodeling in augmented input space.” Comput. Methods Appl. Mech. Eng. 315: 369–395. https://doi.org/10.1016/j.cma.2016.10.042.
Zhang, P. 1996. “Nonparametric importance sampling.” J. Am. Stat. Assoc. 91 (435): 1245–1253. https://doi.org/10.1080/01621459.1996.10476994.
Zio, E., and N. Pedroni. 2009. “Estimation of the functional failure probability of a thermal-hydraulic passive system by subset simulation.” Nucl. Eng. Des. 239 (3): 580–599. https://doi.org/10.1016/j.nucengdes.2008.11.005.
Information & Authors
Information
Published In
ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems, Part A: Civil Engineering
Volume 4 • Issue 3 • September 2018
Copyright
©2018 American Society of Civil Engineers.
History
Received: Apr 24, 2017
Accepted: Dec 21, 2017
Published online: Apr 30, 2018
Published in print: Sep 1, 2018
Discussion open until: Sep 30, 2018
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.