Comparative Study of Kriging and Support Vector Regression for Structural Engineering Applications
Publication: ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems, Part A: Civil Engineering
Volume 4, Issue 2
Abstract
Metamodeling techniques have been widely used as substitutes for high-fidelity and time-consuming models in various engineering applications. Examples include polynomial chaos expansions, neural networks, kriging, and support vector regression (SVR). This paper attempts to compare the latter two in different case studies so as to assess their relative efficiency on simulation-based analyses. Similarities are drawn between these two metamodel types, leading to the use of anisotropy for SVR. Such a feature is not commonly used in the SVR-related literature. Special care was given to a proper automatic calibration of the model hyperparameters by using an efficient global search algorithm, namely the covariance matrix adaptation–evolution scheme. Variants of these two metamodels, associated with various kernel and autocorrelation functions, were first compared on analytical functions and then on finite element–based models. From the comprehensive comparison, it was concluded that anisotropy in the two metamodels clearly improves their accuracy. In general, anisotropic -SVR with the Matérn kernels was shown to be the most effective metamodel.
Get full access to this article
View all available purchase options and get full access to this article.
References
Aizerman, M. A., Braverman, E. A., and Rozonoer, L. (1964). “Theoretical foundations of the potential function method in pattern recognition learning.” Autom. Remote Control, 25, 821–837.
Asmussen, S., and Glynn, P. W. (2007). Stochastic simulation: Algorithms and analysis, Springer, New York.
Au, S.-K., and Beck, J. L. (2001). “Estimation of small failure probabilities in high dimensions by subset simulation.” Probab. Eng. Mech., 16(4), 263–277.
Bachoc, F. (2013). “Cross validation and maximum likelihood estimations of hyper-parameters of Gaussian processes with model misspecifications.” Comput. Stat. Data Anal., 66, 55–69.
Balesdent, M., Morio, J., and Marzat, J. (2013). “Kriging-based adaptive importance sampling algorithms for rare event estimation.” Struct. Saf., 44, 1–10.
Blatman, G., and Sudret, B. (2008). “Adaptive sparse polynomial chaos expansions—Application to structural reliability.” Proc., 4th Int. ASRANet Colloquium, ASRANet Ltd., Glasgow, U.K.
Blatman, G., and Sudret, B. (2010a). “An adaptive algorithm to build up sparse polynomial chaos expansions for stochastic finite element analysis.” Probab. Eng. Mech., 25(2), 183–197.
Blatman, G., and Sudret, B. (2010b). “Reliability analysis of a pressurized water reactor vessel using sparse polynomial chaos expansions.” Proc., 15th IFIP WG7.5 Conf. on Reliability and Optimization of Structural Systems, D. Straub, L. Esteva, and M. Faber, eds., Taylor & Francis, Munich, Germany, 9–16.
Bompard, M. (2011). “Modèles de substitution pour l’optimisation globale de forme en aérodynamique et méthode locale sans paramétrisation.” Ph.D. thesis, Université de Nice Sophia-Antipolis, Nice, France (in French).
Bourinet, J.-M. (2015). “Reliability assessment with adaptive surrogates based on support vec tor machine regression.” Proc., 1st ECCOMAS Thematic Conf. on Uncertainty Quantification in Computational Sciences and Engineering, G. Stefanou, M. Papadrakakis, and V. Papadopoulos, eds., Institute of Structural Analysis and Antiseismic Research, School of Civil Engineering, National Technical Univ. of Athens, Athens, Greece.
Bourinet, J.-M., Deheeger, F., and Lemaire, M. (2011). “Assessing small failure probabilities by combined subset simulation and support vector machines.” Struct. Saf., 33(6), 343–353.
Box, G. E. P., and Draper, N. R. (1986). Empirical model building and response surface, Wiley, New York.
Broomhead, D., and Lowe, D. (1988). “Multivariable functional interpolation and adaptive networks.” Complex Systems, 2(3), 321–355.
Chang, M.-W., and Lin, C.-J. (2005). “Leave-one-out bounds for support vector regression model selection.” Neural Comput., 17(5), 1188–1222.
Chapelle, O. (2002). “Support vector machines: Principes d’induction, réglage automatique et connaissances a priori.” Ph.D. thesis, Université Pierre et Marie Curie, Paris (in French).
Chapelle, O., Vapnik, V., Bousquet, O., and Mukherje, S. (2002). “Choosing multiple parameters for support vector machines.” Mach. Learn., 46(1), 131–159.
Cherkassky, V., and Mulier, F. (2007). Learning from data: Concepts, theory, and methods, Wiley, New York.
Cortes, C., and Vapnik, V. (1995). “Support vector networks.” Mach. Learn., 20(3), 273–297.
Deheeger, F., and Lemaire, M. (2007). “Support vector machine for efficient subset simulations: 2SMART method.” Proc., 10th Int. Conf. on Applications of Statistics and Probability in Civil Engineering (ICASP10), J. Kanda, T. Takada, and H. Furuta, eds., CRC Press, Boca Raton, FL.
De Rocquigny, E. (2012). Modelling under risk and uncertainty: An introduction to statistical, phenomenological and computational methods, Wiley, Chichester, U.K.
Du, Q., Faber, V., and Gunzburger, M. (1999). “Centroidal Voronoi tessellations: Applications and algorithms.” SIAM Rev., 41(4), 637–676.
Dubourg, V. (2011). “Adaptive surrogate models for reliability analysis and reliability-based design optimization.” Ph.D. thesis, Université Blaise Pascal, Clermont-Ferrand, France.
Dubourg, V., Sudret, B., and Bourinet, J.-M. (2011). “Reliability-based design optimization using kriging and subset simulation.” Struct. Multidiscip. Optim., 44(5), 673–690.
Fang, H., Rais-Rohani, M., Liu, Z., and Horstemeyer, M. F. (2005). “A comparative study of metamodeling method for multiobjective crashworthiness optimization.” Comput. Struct., 83(25–26), 2121–2136.
Franco, J. (2008). “Planification d’expériences numériques en phase exploratoire pour la simulation des phénomènes complexes.” Ph.D. thesis, Ecole Nationale Supérieure des Mines, Saint-Etienne, France (in French).
Ghanem, R. G., and Spanos, P. D. (2003). Stochastic finite elements: A spectral approach, Springer, New York.
Ginsbourger, D. (2009). “Multiples métamodèles pour l’approximation et l’optimisation de fonctions numériques multivariables.” Ph.D. thesis, École des Mines de Saint-Étienne, Saint-Etienne, France (in French).
Gunn, S. (1998). “Support vector machines for classification and regression.”, Univ. of Southampton, Southampton, U.K.
Hansen, N. (2005). The CMA evolution strategy: A tutorial, Institut National de Recherche en Informatique et en Automatique, Rocquencourt, France.
Hansen, N., and Kern, S. (2004). “Evaluating the CMA-evolution strategy on multimodal test functions.” Parallel problem solving from nature—PPSN VIII, Vol. 3242, Springer, Berlin, 282–291.
Hansen, N., and Ostermeier, A. (2001). “Completely derandomized self-adaptation in evolution strategies.” Evol. Comput., 9(2), 159–195.
Hurtado, J. E. (2007). “Filtered importance sampling with support vector margin: A powerful method for structural reliability analysis.” Struct. Saf., 29(1), 2–15.
Jin, R., Chen, W., and Simpson, T. W. (2000). “Comparative studies of metamodeling techniques under multiple modeling criteria.” Struct. Multidiscip. Optim., 23(1), 1–13.
Jones, D. R. (2001). “A taxonomy of global optimization methods based on response surfaces.” J. Global Optim., 21(4), 345–383.
Jones, D. R., Schonlau, M., and Welch, W. J. (1998). “Efficient global optimization of expensive black-box functions.” J. Global Optim., 13(4), 455–492.
Kleijnen, J. P. C. (2007). Design and analysis of simulation experiments, 1st Ed., Springer, Berlin.
Koehler, J. R., and Owen, A. (1996). “Computer experiments.” Handbook of statistics, S. Ghosh and C. Rao, eds., Vol. 13, Elsevier Science, Amsterdam, Netherlands, 261–308.
Krige, D. G. (1951). “A statistical approach to some basic mine valuation problems on the Witwatersrand.” J. Chem. Metal. Mining Soc. South Afr., 52(6), 119–139.
Li, Y. F., Ng, S. H., Xie, M., and Goh, T. N. (2010). “A systematic comparison of metamodeling techniques for simulation optimization in decision support systems.” Appl. Soft Comput., 10(4), 1257–1273.
Liu, P.-L., and Der Kiureghian, A. (1991). “Optimization algorithms for structural reliability.” Struct. Saf., 9(3), 161–177.
Lophaven, S. N., Nielsen, H. B., and Sondergaard, J. (2002a). DACE—A MATLAB kriging toolbox—Version 2.0, Informatics and Mathematical modeling Technical Univ. of Denmark, Lyngby, Denmark.
Lophaven, S. N., Nielsen, H. B., and Sondergaard, J. (2002b). “Aspects of the MATLAB toolbox DACE.”, Informatics and Mathematical modeling Technical Univ. of Denmark, Lyngby, Denmark.
Marelli, S., and Sudret, B. (2014). “UQLab: A framework for uncertainty quantification in MATLAB.” Proc., 2nd Int. Conf. on Vulnerability, Risk and Analysis Management (ICVRAM2014), Beer, M., Su, S.-K. and Hall, J. W., eds., Institute for Risk and Uncertainty and the Virtual Engineering Centre of the Univ. of Liverpool, Liverpool, U.K., 2554–2563.
Marrel, A., Iooss, B., van Dorpe, F., and Volkova, E. (2008). “An efficient methodology for modeling complex computer codes with Gaussian processes.” Comput. Stat. Data Anal., 52(10), 4731–4744.
Matheron, G. (1971). La théorie des variables régionalisées et ses applications, Ecole Nationale Superieure de Mines de Paris, Paris.
MATLAB [Computer software]. MathWorks, Natick, MA.
McCulloch, W. S., and Pitts, W. (1988). “Neurocomputing: foundations of research.” Chapter A, Logical calculus of the ideas immanent in nervous activity, MIT Press, Cambridge, MA, 15–27.
McKay, M. D., Beckman, R. J., and Conover, W. J. (1979). “A comparison of three methods for selecting values of input variables in the analysis of output from a computer code.” Technometrics, 42(1), 55–61.
Mullur, A. A., and Messac, A. (2006). “Metamodeling using extended radial basis functions: A comparative approach.” Eng. Comput., 21(3), 203–217.
Nik, M. A., Fayazbakhsh, K., Pasini, D., and Lessard, L. (2014). “A comparative study of metamodelling methods for the design optimization of variable stiffness composites.” Compos. Struct., 107, 494–501.
O’Hagan, A. (2006). “Bayesian analysis of computer code outputs: A tutorial.” Reliab. Eng. Syst. Saf., 91(10–11), 1290–1300.
Picheny, V., Ginsbourger, D., Roustant, O., Haftka, R. T., and Kim, N. H. (2010). “Adaptive designs of experiments for accurate approximation of a target region.” J. Mech. Des., 132(7), 071008.
Rasmussen, C. E., and Williams, C. K. I. (2006). Gaussian processes for machine learning (adaptive computation and machine learning), Massachusetts Institute of Technology, Cambridge, MA.
Roustant, O., Ginsbourger, D., and Deville, Y. (2012). “DiceKriging, DiceOptim: Two R packages for the analysis of computer experiments by kriging-based metamodeling and optimization.” J. Stat. Software, 51(1), 1–55.
Roux, W., Stander, N., Günther, F., and Müllerschön, H. (2006). “Stochastic analysis of highly non-linear structures.” Int. J. Numer. Methods Eng., 65(8), 1221–1242.
Sacks, J., Welch, W., Mitchell, T., and Wynn, H. (1989). “Design and analysis of computer experiments.” Stat. Sci., 4(4), 409–423.
Santner, T., Williams, B., and Notz, W. (2003). The design and analysis of computer experiments, Springer, New York.
Sasena, M. J. (2002). “Flexibility and efficiency enhancements for constrained global design optimization with kriging approximations.” Ph.D. thesis, Univ. of Michigan, Ann Arbor, MI.
Smola, A. J., and Schölkopf, B. (2004). “A tutorial on support vector regression.” Stat. Comput., 14(3), 199–222.
Sobol, I. M. (1967). “On the distribution of points in a cube and the approximate evaluation of integrals.” USSR Comput. Math., 16(3), 784–802.
Sudret, B. (2007). Uncertainty propagation and sensitivity analysis in mechanical models: Contributions to structural reliability and stochastic spectral methods, Université Blaise Pascal, Clermont-Ferrand, France.
Thole, C.-A., and Mei, L. (2003). “Reasons for scatter in crash simulations results.” 4th European LS-DYNA Users Conf., DYNAMore, Stuttgart, Germany.
Vapnik, V. (1995). The nature of statistical learning theory, Springer, New York.
Vapnik, V. (1998). Statistical learning theory, Wiley, New York.
Vapnik, V., and Chapelle, O. (2000). “Bounds on error expectation for support vector machines.” Neural Comput., 12(9), 2013–2036.
Vazquez, E. (2005). “Modélisation comportementale de systèmes non-linéaires multivariables par méthodes à noyaux et applications.” Ph.D. thesis, Université Paris XI Orsay, Orsay, France.
Wei, D., and Rahman, S. (2007). “Structural reliability analysis by univariate decomposition and numerical integration.” Probab. Eng. Mech., 22(1), 27–38.
Yeh, C.-Y., Huang, C.-W., and Lee, S.-J. (2011). “A multiple-kernel support vector regression approach for stock market price forecasting.” Expert Syst. Appl., 38(3), 2177–2186.
Zhao, D., and Xue, D. (2010). “A comparative study of metamodeling methods considering sample quality merits.” Struct. Multidiscip. Optim., 42(6), 923–938.
Information & Authors
Information
Published In
ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems, Part A: Civil Engineering
Volume 4 • Issue 2 • June 2018
Copyright
©2018 American Society of Civil Engineers.
History
Received: Dec 9, 2016
Accepted: Aug 31, 2017
Published online: Jan 19, 2018
Published in print: Jun 1, 2018
Discussion open until: Jun 19, 2018
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.