A Comparative Study of Loss-Level Conservation Constraints in PINNs and Scheme-Level Structure-Preserving Approaches for Hamiltonian Systems
DOI:
https://doi.org/10.54097/zker2m50Keywords:
Physical Neural Networks, Structure Preservation, Conservation Constraints, Hamiltonian Systems, Symplectic Structure, Energy BoundednessAbstract
Physics-Informed Neural Networks (PINNs) provide a neural network-based, mesh-free computational method for solving partial differential equations (PDEs) by incorporating PDE residuals, initial conditions, and boundary conditions into the loss function. However, standard PINNs may still suffer from issues such as conservation drift, insufficient training stability, and difficulty in preserving physical structure when dealing with long-term predictions and complex nonlinear problems. Centered on the core concept of “structure preservation” in scientific computing, this paper systematically compares methods for enhancing physical conservation constraints (Conservation-PINN) in PDE solving with methods for preserving geometric structure (symplex integrators) in the evolution of Hamiltonian systems. This paper selects the heat conduction equation, the linear advection equation, the Burgers equation, and the double-pendulum Hamiltonian system as test cases to conduct a comparative analysis of the basic PINN, Res-PINN, Conservation-PINN, as well as the RK4 method and various symplectic integration methods (including Implicit Midpoint and Gauss-Legendre IRK4).The results show that introducing a conservation loss term can reduce mass drift to some extent during the solution of partial differential equations, and residual connection can improve the training stability of deep PINNs; in Hamiltonian systems, high-order non-symplectic methods (RK4) exhibit excellent local accuracy, while symplectic integrators, by preserving the system’s symplectic structure, can effectively suppress cumulative energy divergence during long-term evolution. This paper provides a reference for understanding the connections and differences between loss function constraints in scientific machine learning and classical geometric numerical integration.
Downloads
References
[1] Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 686 707. https://doi.org/10.1016/j.jcp.2018.10.045.
[2] Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., & Yang, L. (2021). Physics informed machine learning. Nature Reviews Physics, 3(6), 422 440. https:// doi. org/ 10. 1038/s42254 021 00314 5.
[3] Lu, L., Meng, X., Mao, Z., & Karniadakis, G. E. (2021). DeepXDE: A deep learning library for solving differential equations. SIAM Review, 63(1), 208 228. https://doi.org/ 10. 1137/ 19M1274067.
[4] Pang, G., Lu, L., & Karniadakis, G. E. (2019). fPINNs: Fractional physics informed neural networks. SIAM Journal on Scientific Computing, 41(4), A2603 A2626. https://doi.org/ 10. 1137/ 18M1220934.
[5] Zhang, D., Lu, L., Guo, L., & Karniadakis, G. E. (2019). Quantifying total uncertainty in physics informed neural networks for solving forward and inverse stochastic problems. Journal of Computational Physics, 397, 108850. https://doi. org/ 10.1016/j.jcp.2019.108850.
[6] Geneva, N., & Zabaras, N. (2020). Modeling the dynamics of PDE systems with physics constrained deep learning. Journal of Computational Physics, 403, 109056. https://doi.org/ 10. 1016/ j.jcp.2019.109056.
[7] Sirignano, J., & Spiliopoulos, K. (2018). DGM: A deep learning algorithm for solving partial differential equations. Journal of Computational Physics, 375, 1339 1364. https://doi. org/ 10.1016/j.jcp.2018.08.029.
[8] Cranmer, M., Greydanus, S., Hoyer, S., & Ho, J. (2020). Lagrangian neural networks. arXiv preprint arXiv:2003.04630.
[9] Greydanus, S., Dzamba, M., & Yosinski, J. (2019). Hamiltonian neural networks. Advances in Neural Information Processing Systems, 32, 6653 6663.
[10] Chen, R. T. Q., Rubanova, Y., Bettencourt, J., & Duvenaud, D. K. (2018). Neural ordinary differential equations. Advances in Neural Information Processing Systems, 31, 6571 6583.
[11] Wang, Z., & Yu, B. (2022). Long term stability analysis of physics informed neural networks. Computer Methods in Applied Mechanics and Engineering, 390, 114412. https:// doi. org/ 10.1016/j.cma.2021.114412.
[12] Liu, X., & Zhang, Y. (2021). Convergence rates and error bounds for PINNs in dynamical systems. Journal of Scientific Computing, 88(2), 45. https://doi. org/ 10. 1007/ s109 15 021 01552 5.
[13] Smith, J., & Johnson, A. (2021). Boundary enforcement in physics constrained deep learning. IEEE Transactions on Neural Networks and Learning Systems, 32(8), 3450 3462. https:// doi.org/10.1109/TNNLS.2020.3024411.
[14] Jagtap, A. D., Kharazmi, E., & Karniadakis, G. E. (2020). Conservative physics informed neural networks on discrete domains for conservation laws: cPINNs. Computer Methods in Applied Mechanics and Engineering, 365, 113028. https://doi. org/ 10.1016/j.cma.2020.113028.
[15] Bischof, R., & Kraus, M. (2021). Soft and hard constraints in physics informed neural networks. Computer Methods in Applied Mechanics and Engineering, 380, 113787. https:// doi. org/ 10.1016/j.cma.2021.113787.
[16] Jagadish, C., Saha, A., & Das, S. (2022). Manifold constrained neural architectures for constrained dynamical systems. Neurocomputing, 485, 120 132. https://doi.org/10. 1016/j. neucom. 2022.02.017.
[17] Wang, S., Teng, Y., & Perdikaris, P. (2021). Understanding and mitigating gradient pathologies in physics informed neural networks. SIAM Journal on Scientific Computing, 43(5), A3055 A3081. https://doi.org/10.1137/20M1351843.
[18] Wang, S., Sankaran, S., & Perdikaris, P. (2022). Respecting causality for training physics informed neural networks. Computer Methods in Applied Mechanics and Engineering, 398, 115277. https://doi.org/10.1016/j.cma.2022.115277.
[19] Sun, L., Gao, H., Pan, S., & Wang, J. X. (2020). Surrogate modeling for fluid flows based on physics constrained deep learning. Physical Review Fluids, 5(9), 094311. https://doi. org/ 10.1103/PhysRevFluids.5.094311.
[20] Zhang, P. W., & Li, T. J. (2018). Introduction to scientific computing. Higher Education Press.
[21] Feng, K., & Qin, M. Z. (2003). Geometric numerical methods for Hamiltonian systems. Zhejiang Science and Technology Press.
[22] Hairer, E., Lubich, C., & Wanner, G. (2006). Geometric numerical integration: Structure preserving algorithms for ordinary differential equations (2nd ed.). Springer.
[23] Sanz Serna, J. M., & Calvo, M. P. (1994). Numerical Hamiltonian problems. Chapman & Hall.
[24] Leimkuhler, B., & Reich, S. (2004). Simulating Hamiltonian dynamics. Cambridge University Press.
[25] McLachlan, R. I., & Quispel, G. R. W. (2002). Geometric integrators for ODEs. Journal of Physics A: Mathematical and General, 35(18), 3985 4051. https://doi.org/ 10. 1088/ 0305 4470/35/18/303.
[26] Zhong, Y. D., Dey, B., & Chakraborty, A. (2020). Symplectic ODE Net: Learning Hamiltonian dynamics with exact conservation laws. In International Conference on Learning Representations.
[27] Jin, P., Zhang, Z., Kevrekidis, I. G., & Karniadakis, G. E. (2022). Learning Poisson systems and trajectories of autonomous systems via neural networks. IEEE Transactions on Neural Networks and Learning Systems, 33(7), 2980 2991. https:// doi.org/10.1109/TNNLS.2021.3110340.
[28] Chen, Z., Zhang, J., Arjovsky, M., & Bottou, L. (2021). Symplectic networks: Intrinsic structure preserving networks for Hamiltonian systems. Neural Networks, 138, 12 25. https:// doi. org/10.1016/j.neunet.2021.02.008.
[29] Lutter, M., Ritter, C., & Peters, J. (2019). Deep Lagrangian networks: Encountering physics in Euler Lagrange dynamics. In International Conference on Learning Representations.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Frontiers in Computing and Intelligent Systems
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
