RUNNs: Ritz--Uzawa Neural Networks for Solving Variational Problems

Solving partial differential equations (PDEs) using neural networks presents different challenges, including integration errors and spectral bias, often leading to poor approximations. In addition, standard neural network-based methods, such as physics-informed neural networks (PINNs), often fail when dealing with PDEs characterized by low-regularity solutions, which are incompatible with the strong formulation given by PINNs.To address these limitations, we introduce the Ritz--Uzawa neural networks (RUNNs) framework, an iterative methodology to solve variational problems, such as strong, weak, and ultra-weak formulations. Rewriting the PDE as a sequence of Ritz-type minimization problems within a Uzawa framework provides an iterative method that, in specific cases, reduces variance of the numerical integration error during training. We demonstrate that the strong formulation offers a passive variance reduction mechanism, whereas variance remains persistent in weak and ultra-weak formulations. Furthermore, we address the spectral bias of standard architectures through a data-driven frequency tuning strategy. By initializing a sinusoidal Fourier feature mapping based on the normalized cumulative power spectral density (NCPSD) of previous residuals or their proxies, the network dynamically adapts its frequency modes to capture high-frequency components and severe singularities. Numerical experiments demonstrate that RUNNs accurately solve highly oscillatory solutions and successfully recover a discontinuous $L^2$ solution from a distributional $H^{-2}$ source -- a scenario that is incompatible with an $H^1$-formulation.