Characterizing Parameter Equivalence and Approximation in 1D FEM and ReLU NNs

Abstract This paper introduces a novel methodology for characterizing parameter equivalence between Rectified Linear Unit (ReLU) Neural Networks (NNs) and 1D Finite Element Models (FEMs). By expressing the FEM solution equation in terms of neural networks and employing a hybrid scheme involving partially trained nested neural networks, we bridge the gap between these two computational paradigms. Our approach offers a promising avenue to leverage the flexibility of neural networks while retaining the established principles of FEM. Through extensive numerical experiments, we demonstrate the efficacy of our scheme in function interpolation and solving partial differential equations (PDEs). In function interpolation tasks, our hybrid scheme consistently outperforms fully trainable ReLU NNs, achieving significantly lower error factors across various scenarios. Similarly, in solving PDEs, our approach exhibits superior accuracy compared to fully trainable ReLU NNs, as evidenced by lower L2-error factors. We further extend the application of our hybrid scheme to diverse PDEs including the heat, Poisson, and wave equations, showcasing its versatility and effectiveness across different problem domains. This work not only presents a significant advancement in computational methods but also lays the groundwork for redefining the FEM algorithm within an algorithmic framework. These findings have implications for a wide range of scientific and engineering disciplines, and we believe they warrant consideration for publication in high-impact journals.