Document Type : Research Article

Authors

1 Professor, Department of Aerospace Engineering at Sharif University of Technology, Tehran. Iran

2 Ph. D. Student, Department of Aerospace Engineering at Sharif University of Technology, Tehran. Iran

Abstract

Development of Physics-Informed Neural Networks (PINNs) as nonlinear dynamics surrogates is investigated. PINNs are unsupervised neural networks in which the input-output relationship is established via a specific dynamic relationship (differential equation). In this regard, the derivatives are determined by utilizing Automatic Differentiation over the network’s graph. Hence, PINNs can be utilized to build complex surrogates for nonlinear dynamical systems which can later be used in real-time control applications. In this study, it is shown that PINNs can adequately capture the dynamics investigated. Even in regions of the state space where there are no training sample points, a PINN surrogate provides an acceptable approximation of the dynamical system. To investigate the hypothesis, three categories of nonlinear dynamics are examined: (1) self-sustained, (2) excitatory, and (3) chaotic systems. As implied by the results, PINNs can estimate self-sustaining and chaotic systems with sufficient accuracy. However, the concept is not as successful with excitatory dynamics that mandates further detailed studies on these surrogates.

Keywords

Main Subjects

[1] J. Guckenheimer and P. Holmes, Nonlinear oscillations, dynamical systems, and bifurcations of vector fields. Springer Science & Business Media, 2013.
[2] H. Voss, M. Bünner, and M. Abel, "Identification of continuous, spatiotemporal systems," Physical Review E, vol. 57, no. 3, p. 2820, 1998.
[3] M. Quade, M. Abel, K. Shafi, R. K. Niven, and B. R. Noack, "Prediction of dynamical systems by symbolic regression," Physical Review E, vol. 94, no. 1, p. 012214, 2016.
[4] M. Schmidt and H. Lipson, "Distilling free-form natural laws from experimental data," science, vol. 324, no. 5923, pp. 81-85, 2009.
[5] J. Pathak, B. Hunt, M. Girvan, Z. Lu, and E. Ott, "Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach," Physical review letters, vol. 120, no. 2, p. 024102, 2018.
[6] J. Pathak, Z. Lu, B. R. Hunt, M. Girvan, and E. Ott, "Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data," Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 27, no. 12, p. 121102, 2017.
[7] I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep learning (no. 2). MIT press Cambridge, 2016.
[8] R. S. Zimmermann and U. Parlitz, "Observing spatio-temporal dynamics of excitable media using reservoir computing," Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 28, no. 4, p. 043118, 2018.
[9] Z. Lu, B. R. Hunt, and E. Ott, "Attractor reconstruction by machine learning," Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 28, no. 6, p. 061104, 2018.
[10] R. Cestnik and M. Abel, "Inferring the dynamics of oscillatory systems using recurrent neural networks," Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 29, no. 6, p. 063128, 2019.
[11] B. A. Pearlmutter, "Learning state space trajectories in recurrent neural networks," Neural Computation, vol. 1, no. 2, pp. 263-269, 1989.
[12] K.-i. Funahashi and Y. Nakamura, "Approximation of dynamical systems by continuous time recurrent neural networks," Neural networks, vol. 6, no. 6, pp. 801-806, 1993.
[13] C. A. Bailer-Jones, D. J. MacKay, and P. J. Withers, "A recurrent neural network for modelling dynamical systems," network: computation in neural systems, vol. 9, no. 4, pp. 531-547, 1998.
[14] D. Svozil, V. Kvasnicka, and J. Pospichal, "Introduction to multi-layer feed-forward neural networks," Chemometrics and intelligent laboratory systems, vol. 39, no. 1, pp. 43-62, 1997.
[15] M. Raissi, P. Perdikaris, and G. E. Karniadakis, "Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations," Journal of Computational Physics, vol. 378, pp. 686-707, 2019.
[16]  H. Gomi and M. Kawato, "Neural network control for a closed-loop system using feedback-error-learning," Neural Networks, vol. 6, no. 7, pp. 933-946, 1993.
[17]  J. Sarangapani, Neural network control of nonlinear discrete-time systems. CRC press, 2018.
[18]  F. Arnold and R. King, "State–space modeling for control based on physics-informed neural networks," Engineering Applications of Artificial Intelligence, vol. 101, p. 104195, 2021.
[19] S. Hochreiter and J. Schmidhuber, "Long Short-Term Memory," Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997, doi: 10.1162/neco.1997.9.8.1735.