Machine Learning Approach to Model Order Reduction of Nonlinear Systems via Autoencoder and LSTM Networks
Publication: Journal of Engineering Mechanics
Volume 147, Issue 10
Abstract
In analyzing and assessing the condition of dynamical systems, it is necessary to account for nonlinearity. Recent advances in computation have rendered previously computationally infeasible analyses readily executable on common computer hardware. However, in certain use cases, such as uncertainty quantification or high precision real-time simulation, the computational cost remains a challenge. This necessitates the adoption of reduced-order modeling methods, which can reduce the computational toll of such nonlinear analyses. In this work, we propose a reduction scheme relying on the exploitation of an autoencoder as means to infer a latent space from output-only response data. This latent space, which in essence approximates the system’s nonlinear normal modes (NNMs), serves as an invertible reduction basis for the nonlinear system. The proposed machine learning framework is then complemented via the use of long short-term memory (LSTM) networks in the reduced space. These are used for creating a nonlinear reduced-order model (ROM) of the system, able to recreate the full system’s dynamic response under a known driving input.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This work was carried out as part of the Initial Training Networks (ITN) project DyVirt and has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under the Marie Skłodowska-Curie Grant Agreement No. 764547.
References
Ansys. 2013. Ansys mechanical APDL theory reference. 15th ed. Canonsburg, PA: Ansys.
Baldi, P., and K. Hornik. 1989. “Neural networks and principal component analysis: Learning from examples without local minima.” Neural Networks 2 (1): 53–58. https://doi.org/10.1016/0893-6080(89)90014-2.
Bayer, J. S. 2015. “Learning sequence representations.” Ph.D. thesis, Dept. of Informatics, Technische Universität München.
Bengio, Y., and Y. Lecun. 2007. Scaling learning algorithms towards AI. Cambridge, MA: MIT Press.
Bengio, Y., P. Simard, and P. Frasconi. 1994. “Learning long-term dependencies with gradient descent is difficult.” IEEE Trans. Neural Networks 5 (2): 157–166. https://doi.org/10.1109/72.279181.
Besselink, B., U. Tabak, A. Lutowska, N. van de Wouw, H. Nijmeijer, D. J. Rixen, M. E. Hochstenbach, and W. H. A. Schilders. 2013. “A comparison of model reduction techniques from structural dynamics, numerical mathematics and systems and control.” J. Sound Vib. 332 (19): 4403–4422. https://doi.org/10.1016/j.jsv.2013.03.025.
Brownlee, J. 2017. Long short-term memory networks with python: Develop sequence prediction models with deep learning. Vermont, VIC: Jason Brownlee.
Carlberg, K., and C. Farhat. 2008. “A compact proper orthogonal decomposition basis for optimization-oriented reduced-order models.” In Proc., 12th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conf. Victoria, BC, Canada: American Institute of Aeronautics and Astronautics.
Chatzi, E. N., and A. W. Smyth. 2009. “The unscented Kalman filter and particle filter methods for nonlinear structural system identification with non-collocated heterogeneous sensing.” Struct. Control Health Monit. 16 (1): 99–123. https://doi.org/10.1002/stc.290.
Chatzi, E. N., A. W. Smyth, and S. F. Masri. 2010. “Experimental application of on-line parametric identification for nonlinear hysteretic systems with model uncertainty.” Struct. Saf. 32 (5): 326–337. https://doi.org/10.1016/j.strusafe.2010.03.008.
Chopra, A. K. 2012. Dynamics of structures. Harlow, UK: Pearson.
Craig, R. R., Jr., and M. C. C. Bampton. 1968. “Coupling of substructures for dynamic analyses.” AIAA J. 6 (7): 1313–1319. https://doi.org/10.2514/3.4741.
De Klerk, D., D. J. Rixen, and S. N. Voormeeren. 2008. “General framework for dynamic substructuring: History, review and classification of techniques.” AIAA J. 46 (5): 1169–1181. https://doi.org/10.2514/1.33274.
Dervilis, N., T. E. Simpson, D. J. Wagg, and K. Worden. 2019. “Nonlinear modal analysis via non-parametric machine learning tools.” Strain 55 (1): e12297. https://doi.org/10.1111/str.12297.
Ebrahimian, H., R. Astroza, J. P. Conte, and R. A. de Callafon. 2017. “Nonlinear finite element model updating for damage identification of civil structures using batch Bayesian estimation.” Mech. Syst. Sig. Process. 84 (Feb): 194–222. https://doi.org/10.1016/j.ymssp.2016.02.002.
Figueiredo, E., G. Park, J. Figueiras, C. Farrar, and K. Worden. 2009. Structural health monitoring algorithm comparisons using standard data sets. Los Alamos, NM: Los Alamos National Laboratory.
Graves, A., S. Fernández, M. Liwicki, H. Bunke, and J. Schmidhuber. 2007. “Unconstrained online handwriting recognition with recurrent neural networks.” In Proc., 20th Int. Conf. on Neural Information Processing Systems, 577–584. Victoria, BC, Canada: American Institute of Aeronautics and Astronautics.
Haller, G., and S. Ponsioen. 2016. “Nonlinear normal modes and spectral submanifolds: Existence, uniqueness and use in model reduction.” Nonlinear Dyn. 86 (3): 1493–1534. https://doi.org/10.1007/s11071-016-2974-z.
Hinton, G. E., and R. R. Salakhutdinov. 2006. “Reducing the dimensionality of data with neural networks.” Science 313 (5786): 504–507. https://doi.org/10.1126/science.1127647.
Hochreiter, S., and J. Schmidhuber. 1997. “Long short-term memory.” Neural Comput. 9 (8): 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735.
Holden, D., J. Saito, T. Komura, and T. Joyce. 2015. Learning motion manifolds with convolutional autoencoders. Kobe, Japan: American Institute of Aeronautics and Astronautics.
Ismail, M., F. Ikhouane, and J. Rodellar. 2009. “The hysteresis Bouc-Wen model, A survey.” Arch. Comput. Methods Eng. 16 (2): 161–188. https://doi.org/10.1007/s11831-009-9031-8.
Jain, S., P. Tiso, and G. Haller. 2018. “Exact nonlinear model reduction for a von Kármán beam: Slow-fast decomposition and spectral submanifolds.” J. Sound Vib. 423 (Jun): 195–211. https://doi.org/10.1016/j.jsv.2018.01.049.
Joldes, G. R., A. Wittek, and K. Miller. 2010. “Real-time nonlinear finite element computations on GPU—Application to neurosurgical simulation.” Comput. Methods Appl. Mech. Eng. 199 (49–52): 3305–3314. https://doi.org/10.1016/j.cma.2010.06.037.
Kerschen, G., J.-C. Golinval, A. F. Vakakis, and L. A. Bergman. 2005. “The method of proper orthogonal decomposition for dynamical characterization and order reduction of mechanical systems: An overview.” Nonlinear Dyn. 41 (1): 147–169. https://doi.org/10.1007/s11071-005-2803-2.
Kerschen, G., M. Peeters, J.-C. Golinval, and A. F. Vakakis. 2009. “Nonlinear normal modes, Part I: A useful framework for the structural dynamicist.” Mech. Syst. Sig. Process. 23 (1): 170–194. https://doi.org/10.1016/j.ymssp.2008.04.002.
Kingma, D. P., and L. J. Ba. 2015. “Adam: A method for stochastic optimization.” In Proc., 3rd Int. Conf. on Learning Representations, ICLR 2015—Conf. Track. Ithaca, NY: International Conference on Learning Representations.
Kingma, D. P., and M. Welling. 2014. “Auto-encoding variational bayes.” In Proc., 2nd Int. Conf. on Learning Representations, ICLR 2014—Conf. Ithaca, NY: International Conference on Learning Representations.
Kuether, R. J., B. J. Deaner, J. J. Hollkamp, and M. S. Allen. 2015. “Evaluation of geometrically nonlinear reduced-order models with nonlinear normal modes.” AIAA J. 53 (11): 3273–3285. https://doi.org/10.2514/1.J053838.
Ladjal, S., A. Newson, and C.-H. Pham. 2019. “A PCA-like autoencoder.” Preprint, submitted April 2, 2019. http://arxiv.org/abs/1904.01277.
Lagaros, N. D., and M. Papadrakakis. 2012. “Neural network based prediction schemes of the non-linear seismic response of 3D buildings.” Adv. Eng. Software 44 (1): 92–115. https://doi.org/10.1016/j.advengsoft.2011.05.033.
Lee, K., and K. T. Carlberg. 2020. “Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders.” J. Comput. Phys. 404 (Mar): 108973. https://doi.org/10.1016/j.jcp.2019.108973.
Li, T., T. Wu, and Z. Liu. 2020. “Nonlinear unsteady bridge aerodynamics: Reduced-order modeling based on deep LSTM networks.” J. Wind Eng. Ind. Aerodyn. 198 (Mar): 104116. https://doi.org/10.1016/j.jweia.2020.104116.
Liu, X., D. J. Wagg, and S. A. Neild. 2019. “An explanation for why natural frequencies shifting in structures with membrane stresses, using backbone curve models.” In Vol. 1 of Nonlinear dynamics, edited by G. Kerschen, 9–19. Cham, Switzerland: Springer.
Miah, M. S., E. N. Chatzi, V. K. Dertimanis, and F. Weber. 2015. “Nonlinear modeling of a rotational MR damper via an enhanced Bouc–Wen model.” Smart Mater. Struct. 24 (10): 105020. https://doi.org/10.1088/0964-1726/24/10/105020.
Noël, J.-P., and G. Kerschen. 2017. “Nonlinear system identification in structural dynamics: 10 more years of progress.” Mech. Syst. Sig. Process. 83 (Jan): 2–35. https://doi.org/10.1016/j.ymssp.2016.07.020.
Olah, C. 2015. “Understanding LSTM networks. colah’s blog.” Accessed May 28, 2020. https://colah.github.io/posts/2015-08-Understanding-LSTMs/.
Papadimitriou, C., and D.-C. Papadioti. 2013. “Component mode synthesis techniques for finite element model updating.” Comput. Struct. 126 (Sep): 15–28. https://doi.org/10.1016/j.compstruc.2012.10.018.
Pesheck, E., C. Pierre, and S. W. Shaw. 2001. “Accurate reduced-order models for a simple rotor blade model using nonlinear normal modes.” Math. Comput. Modell. 33 (10–11): 1085–1097. https://doi.org/10.1016/S0895-7177(00)00301-0.
Raissi, M., P. Perdikaris, and G. E. Karniadakis. 2019. “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.” J. Comput. Phys. 378 (Feb): 686–707. https://doi.org/10.1016/j.jcp.2018.10.045.
Rawat, W., and Z. Wang. 2017. “Deep convolutional neural networks for image classification: A comprehensive review.” Neural Comput. 29 (9): 2352–2449. https://doi.org/10.1162/neco_a_00990.
Rixen, D. J. 2004. “A dual Craig–Bampton method for dynamic substructuring.” J. Comput. Appl. Math. 168 (1): 383–391. https://doi.org/10.1016/j.cam.2003.12.014.
Rogers, T. J., T. B. Schön, A. Lindholm, K. Worden, and E. J. Cross. 2019. “Identification of a duffing oscillator using particle Gibbs with ancestor sampling.” J. Phys. Conf. Ser. 1264 (1): 012051. https://doi.org/10.1088/1742-6596/1264/1/012051.
Rosenberg, R. M. 1962. “The normal modes of nonlinear n-degree-of-freedom systems.” J. Appl. Mech. 29 (1): 7–14. https://doi.org/10.1115/1.3636501.
Roweis, S. T., and L. K. Saul. 2000. “Nonlinear dimensionality reduction by locally linear embedding.” Science 290 (5500): 2323–2326. https://doi.org/10.1126/science.290.5500.2323.
Rumelhart, D. E., and J. L. McClelland. 1987. Learning internal representations by error propagation, 318–362. Cambridge, MA: MIT Press.
Sadhu, A., S. Narasimhan, and J. Antoni. 2017. “A review of output-only structural mode identification literature employing blind source separation methods.” Mech. Syst. Sig. Process. 94 (Sep): 415–431. https://doi.org/10.1016/j.ymssp.2017.03.001.
Shaw, S. W., and C. Pierre. 1993. “Normal modes for non-linear vibratory systems.” J. Sound Vib. 164 (1): 85–124. https://doi.org/10.1006/jsvi.1993.1198.
Smith, M. 2009. ABAQUS/standard user’s manual, version 6.9. Providence, RI: Simulia.
Snaiki, R., and T. Wu. 2019. “Knowledge-enhanced deep learning for simulation of tropical cyclone boundary-layer winds.” J. Wind Eng. Ind. Aerodyn. 194 (Nov): 103983. https://doi.org/10.1016/j.jweia.2019.103983.
Sudret, B., S. Marelii, and J. Wiart. 2017. “Surrogate models for uncertainty quantification: An overview.” In Proc., 2017 11th European Conf. on Antennas and Propagation (EUCAP), 793–797. New York: IEEE.
Tao, F., X. Liu, H. Du, and W. Yu. 2020. “Physics-informed artificial neural network approach for axial compression buckling analysis of thin-walled cylinder.” AIAA J. 58 (6): 2737–2747. https://doi.org/10.2514/1.J058765.
Touzé, C., and M. Amabili. 2006. “Nonlinear normal modes for damped geometrically nonlinear systems: Application to reduced-order modelling of harmonically forced structures.” J. Sound Vib. 298 (4–5): 958–981. https://doi.org/10.1016/j.jsv.2006.06.032.
Vincent, P., H. Larochelle, Y. Bengio, and P.-A. Manzagol. 2008. “Extracting and composing robust features with denoising autoencoders.” In Proc., 25th Int. Conf. on Machine Learning, 1096–1103. New York: Association for Computing Machinery.
Vlachas, K., K. Tatsis, K. Agathos, A. R. Brink, and E. Chatzi. 2020. “A local basis approximation approach for nonlinear parametric model order reduction.” Preprint, submitted March 16, 2020. http://arxiv.org/abs/2003.07716.
Vlachas, P. R., W. Byeon, Z. Y. Wan, T. P. Sapsis, and P. Koumoutsakos. 2018. “Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks.” Proc. R. Soc. London, Ser. A 474 (2213): 20170844. https://doi.org/10.1098/rspa.2017.0844.
Wang, H., and T. Wu. 2020. “Knowledge-enhanced deep learning for wind-induced nonlinear structural dynamic analysis.” J. Struct. Eng. 146 (11): 04020235. https://doi.org/10.1061/(ASCE)ST.1943-541X.0002802.
Wang, Y. 2017. “A new concept using LSTM neural networks for dynamic system identification.” In Proc., of the American Control Conf. (ACC), 5324–5329. Seattle, Washington. New York: IEEE.
Wold, S., K. Esbensen, and P. Geladi. 1987. “Principal component analysis.” Chemom. Intell. Lab. Syst. 2 (1–3): 37–52. https://doi.org/10.1016/0169-7439(87)80084-9.
Worden, K., and P. L. Green. 2017. “A machine learning approach to nonlinear modal analysis.” Mech. Syst. Sig. Process. 84 (Feb): 34–53. https://doi.org/10.1016/j.ymssp.2016.04.029.
Wu, L., and P. Tiso. 2016. “Nonlinear model order reduction for flexible multibody dynamics: A modal derivatives approach.” Multibody Syst. Dyn. 36 (4): 405–425. https://doi.org/10.1007/s11044-015-9476-5.
Wu, R.-T., and M. R. Jahanshahi. 2019. “Deep convolutional neural network for structural dynamic response estimation and system identification.” J. Eng. Mech. 145 (1): 04018125. https://doi.org/10.1061/(ASCE)EM.1943-7889.0001556.
Wu, T., and K. Ahsan. 2011. “Modeling hysteretic nonlinear behavior of bridge aerodynamics via cellular automata nested neural network.” J. Wind Eng. Ind. Aerodyn. 99 (4): 378–388. https://doi.org/10.1016/j.jweia.2010.12.011.
Wu, Y., et al. 2016. “Google’s neural machine translation system: Bridging the gap between human and machine translation.” Preprint, submitted September 26, 2016. http://arxiv.org/abs/1609.08144.
Yoo, Y., S. Yun, H. Jin Chang, Y. Demiris, and J. Young Choi. 2017. “Variational autoencoded regression: High dimensional regression of visual data on complex manifold.” In Proc., 2017 IEEE Conf. on Computer Vision and Pattern Recognition, 3674–3683. New York: IEEE.
Zhang, R., Z. Chen, S. Chen, J. Zheng, O. Büyüköztürk, and H. Sun. 2019. “Deep long short-term memory networks for nonlinear structural seismic response prediction.” Comput. Struct. 220 (Aug): 55–68. https://doi.org/10.1016/j.compstruc.2019.05.006.
Zhang, R., Y. Liu, and H. Sun. 2020. “Physics-informed multi-LSTM networks for metamodeling of nonlinear structures.” Comput. Methods Appl. Mech. Eng. 369 (Sep): 113226. https://doi.org/10.1016/j.cma.2020.113226.
Information & Authors
Information
Published In
Journal of Engineering Mechanics
Volume 147 • Issue 10 • October 2021
Copyright
© 2021 American Society of Civil Engineers.
History
Received: Sep 3, 2020
Accepted: Apr 6, 2021
Published online: Jul 19, 2021
Published in print: Oct 1, 2021
Discussion open until: Dec 19, 2021
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Thomas Simpson, Nikolaos Dervilis, Philippe Couturier, Nico Maljaars, Eleni Chatzi, Reduced order modeling of non-linear monopile dynamics via an AE-LSTM scheme, Frontiers in Energy Research, 10.3389/fenrg.2023.1128201, 11, (2023).
- A. Rostamijavanani, Shanwu Li, Yongchao Yang, Data-Driven Modeling of Parameterized Nonlinear Dynamical Systems with a Dynamics-Embedded Conditional Generative Adversarial Network, Journal of Engineering Mechanics, 10.1061/JENMDT.EMENG-7038, 149, 11, (2023).
- Shanwu Li, Yongchao Yang, Efficient Data-Driven Modeling of Nonlinear Dynamical Systems via Metalearning, Journal of Engineering Mechanics, 10.1061/JENMDT.EMENG-6917, 149, 3, (2023).
- Necati Catbas, Onur Avci, A review of latest trends in bridge health monitoring, Proceedings of the Institution of Civil Engineers - Bridge Engineering, 10.1680/jbren.21.00093, (1-16), (2022).
- Ang Zhao, Yanhua Ma, Research on Recommendation of Big Data for Higher Education Based on Deep Learning, Scientific Programming, 10.1155/2022/5448442, 2022, (1-8), (2022).
- Rong Dou, Fansheng Meng, Prophet-LSTM combination model carbon trading price prediction research, Second International Symposium on Computer Technology and Information Science (ISCTIS 2022), 10.1117/12.2653785, (81), (2022).
- Zhilu Lai, Wei Liu, Xudong Jian, Kiran Bacsa, Limin Sun, Eleni Chatzi, Neural modal ordinary differential equations: Integrating physics-based modeling with neural ordinary differential equations for modeling high-dimensional monitored structures, Data-Centric Engineering, 10.1017/dce.2022.35, 3, (2022).
- Ihsan Engin Bal, Eleni Smyrou, Implementation of Emerging Technologies in Seismic Risk Estimation, Progresses in European Earthquake Engineering and Seismology, 10.1007/978-3-031-15104-0_17, (279-294), (2022).
- Stefania Fresca, Giorgio Gobat, Patrick Fedeli, Attilio Frangi, Andrea Manzoni, Deep learning‐based reduced order models for the real‐time simulation of the nonlinear dynamics of microstructures, International Journal for Numerical Methods in Engineering, 10.1002/nme.7054, 123, 20, (4749-4777), (2022).
- Konstantinos Vlachas, David Najera-Flores, Carianne Martinez, Adam R. Brink, Eleni Chatzi, A Physics-Based Reduced Order Model with Machine Learning-Boosted Hyper-Reduction, Topics in Modal Analysis & Parameter Identification, Volume 8, 10.1007/978-3-031-05445-7_15, (131-139), (2012).