# Spectral-Informed Neural Networks Outperform Spectral Methods in High-dimensional PDEs **Authors:** Tianchi Yu, Ivan Oseledets **Paper:** [arXiv:2607.13566](https://arxiv.org/abs/2607.13566) · [[Zotero: 3P3FQLN4]] **Status:** ICML 2026 proceedings paper / arXiv v1, 15 July 2026 ## Executive summary This paper proposes **Modified Spectral-Informed Neural Networks (Modified SINNs)** for solving PDEs in moderate and high spatial dimension. The central idea is to have a neural network predict a solution’s **spectral coefficients** rather than its values in physical space, while injecting two pieces of harmonic-analysis prior knowledge: the expected decay of coefficients at high frequency and an embedding of the spectral basis evaluated on a small collocation set. The authors argue that this representation combines useful properties of spectral methods and physics-informed neural networks (PINNs). It avoids differentiating the network with respect to spatial coordinates during training, can enforce some boundary conditions exactly through the choice of basis, and exploits smoothness-induced coefficient decay. In experiments, the method is strongest when the solution has a sparse or structured spectral representation. It outperforms sparse-grid spectral methods when coefficients are missing and gives substantially smaller errors than the selected PINN and Deep Ritz Method (DRM) baselines on the paper’s high-dimensional Poisson and heat benchmarks. The paper’s most important caveat is equally central: its success is demonstrated under **sparse-coefficient assumptions**, with many benchmark solutions having separable or symmetric structure. The authors explicitly state that genuinely dense high-dimensional functions remain difficult for SINNs, PINNs, DRMs, and current spectral methods alike. ## Problem and motivation Spectral methods approximate a PDE solution using a truncated global basis expansion, $ u_N(x,t) = \sum_{k \in K_N} \hat u_k(t)\,\phi_k(x), $ where \(\phi_k\) is, for example, a Fourier or Chebyshev basis function. For smooth solutions these methods can converge extremely rapidly, but their number of required modes and the cost of transforms scale poorly with dimension. Sparse-grid spectral methods (SGSMs) alleviate this only partially: their hyperbolic-cross index sets still become impractical as dimension and resolution grow. PINNs instead represent \(u(x,t)\) directly with a network and minimize PDE, boundary, and initial-condition residuals. They can work in high dimensions, but require automatic differentiation of spatial derivatives and often have limited accuracy or training difficulty for oscillatory/multiscale functions. Original SINNs move the learning problem into the spectral domain: the input is frequency index \(k\) (and time \(t\), when applicable), and the network outputs \(\hat u_\theta(k,t)\). For bases that diagonalize the spatial operator, the PDE residual can be evaluated coefficient-wise. This removes spatial derivatives of the neural network from the loss. The present paper modifies that architecture to make it more stable and better able to fill in uncomputed coefficients. ## Method: Modified SINNs ### 1. Spectral-domain physics loss A SINN predicts coefficients \(\hat u_\theta(k,t)\), rather than a physical-space field. The physical solution can be reconstructed afterward by the inverse spectral sum. The loss enforces the transformed PDE and transformed initial/boundary conditions. For a periodic Poisson equation using Fourier modes, the residual simplifies to a coefficient-wise condition of the form $ \sum_{k\in K_{N_f}} \left\|\|k\|_2^2\,\hat u_\theta(k) + \hat f_k\right\|^2. $ For periodic heat equations, the residual similarly involves \(\partial_t \hat u_\theta(k,t)\) and \(\|k\|_2^2\hat u_\theta(k,t)\). Because the Fourier basis already satisfies periodicity, no separate boundary loss is needed in those examples. ### 2. Learnable coefficient-decay scaling Smoothness predicts the magnitude profile of spectral coefficients. In particular, analytic periodic functions have Fourier coefficients bounded by an exponential decay expression; finitely smooth functions instead exhibit algebraic decay. Modified SINNs multiply the base network output by a learnable decay factor. For the Fourier/analytic case, the architecture uses $ \exp(-\|\alpha\!\cdot\! k\|_1), $ with learnable \(\alpha\). For non-smooth cases, the paper proposes an algebraic factor based on powers of \(|k_j|\). The purpose is to allocate the network’s capacity toward dominant low-frequency modes, suppress implausibly large high-frequency predictions, and stabilize training when frequency samples are sparse. ### 3. Basis embedding The second modification supplies the network with an embedding constructed from known basis evaluations. On a much smaller collocation set \(X_c\), the method feeds the values of \(\phi_k\) through an MLP, producing \(z_a(k)\). Intuitively, this gives the model explicit information about the oscillatory structure associated with each frequency index. The proposed Fourier-case output is $ \hat u_{\mathrm{md}}(k,t)=z_a(k)\,\hat u_\theta(k,t)\,\exp(-\|\alpha\!\cdot\! k\|_1). $ Theorem 3.1 states that, under a rank condition on the embedding’s basis-evaluation matrix and sufficiently small fitting error on the coarse set, the embedding generalizes to unseen coefficients at the same order as the quadrature error. This is a conditional approximation result, not a full end-to-end PDE convergence guarantee. ## Experimental evidence All reported errors are relative \(L^2\) errors. Experiments ran on an NVIDIA A40-40G GPU. ### Recovering missing coefficients The authors first randomly mask spectral coefficients and train on the remainder. With 90% of Fourier coefficients available, the reported missing-coefficient errors were comparable to overall coefficient errors across dimensions 2, 3, 6, and 8. In a 2-D Chebyshev experiment, the paper reports approximately \(10^{-3}\) accuracy when at least 60% of coefficients are available. The practical implication is not that arbitrary missing data can always be recovered, but that the learned spectral representation can interpolate unobserved modes when the coefficient field has exploitable structure. ### Compared with sparse-grid spectral methods On middle-dimensional Poisson, heat, and convection equations, SGSM is more accurate when **all** required coefficients are known. Once coefficients are missing, however, SGSM degrades sharply because it has no mechanism to infer them; SINNs degrade more gradually. A hybrid SGSM+SINN test on time-dependent harmonic-oscillator Schrödinger equations makes this complementarity concrete. SGSM computes available coefficients, then SINN predicts masked ones. With 10–20% of coefficients withheld, this reduced SGSM errors by large factors across dimensions 2, 5, and 8. For example, at dimension 8 and 20% masking, the error changes from \(4.41\times10^{-1}\) for SGSM to \(1.67\times10^{-3}\) for SGSM+SINN. ### Compared with PINNs and DRMs in high dimensions For Poisson equations, the reported SINN errors remain much lower than the selected PINN (PirateNets) and DRM/NDRM baselines as dimension rises: | Dimension | PINN | DRM | SINN | |---:|---:|---:|---:| | 10 | \(9.68\times10^{-3}\) | \(1.03\times10^{-2}\) | \(1.99\times10^{-4}\) | | 30 | \(1.70\times10^{-2}\) | \(2.23\times10^{-2}\) | \(2.76\times10^{-4}\) | | 50 | \(3.17\times10^{-2}\) | \(3.20\times10^{-2}\) | \(1.20\times10^{-3}\) | | 100 | \(5.50\times10^{-1}\) | \(5.94\times10^{-2}\) | \(7.75\times10^{-3}\) | The analogous heat-equation results favor SINN strongly through 50 dimensions, though all methods deteriorate at dimension 100 (SINN: \(3.69\times10^{-1}\)). On a more challenging high-dimensional Poisson construction with sparse, low-frequency-weighted mode sets, SINN is again substantially better in most configurations. Performance eventually declines as the number of valid modes becomes very large, e.g. the 100-D case with 10,000 valid coefficients. ### Ablation The ablation separates the two proposed components. On a 2-D Fourier task with 40% of coefficients missing, the original SINN has relative error \(2.11\times10^{-1}\); adding only coefficient decay changes it little (\(2.06\times10^{-1}\)); adding only the embedding lowers it to \(3.27\times10^{-3}\); using both gives \(3.08\times10^{-3}\). The authors therefore attribute the main missing-mode recovery benefit to basis embedding and the additional numerical improvement to coefficient decay. ### Cost claims For high-dimensional Poisson experiments, the paper reports faster training iteration rates for SINN and lower memory use than the derivative-based PINN/DRM baselines. At dimension 100, reported rates are 14.6 iterations/s for SINN, 3.45 for DRM, and 1.46 for PINN; reported memory is about 997 MB for SINN, versus 2.31 GB for DRM and 6.95 GB for PINN. The tradeoff is somewhat slower inference at high dimension because SINN processes spectral coefficients and reconstructs physical values. ## Interpretation The paper is best read as a **structured function-approximation approach for PDEs**, rather than a general escape from the curse of dimensionality. Its inductive bias is particularly suitable when: - the solution is smooth enough for rapid spectral coefficient decay; - a suitable basis incorporates the geometry/boundary conditions; - only a manageable, sparse set of significant modes is needed; - some coefficients can be computed conventionally and the rest inferred; or - direct spatial derivatives of a neural surrogate are costly or unstable. The hybrid use case is compelling: retain spectral solvers where their coefficients are accurate and use SINNs only to complete a partially known spectrum. ## Limitations and questions 1. **Sparse/structured solutions are essential.** Most strong results use separable or symmetric targets. The authors acknowledge that genuinely dense high-dimensional coefficient fields remain intractable. 2. **Benchmark construction is favorable to spectral access.** In several experiments, analytic coefficients are supplied directly to avoid transform error. In a real physical-space PDE, obtaining those coefficients can itself be expensive in high dimension. 3. **Missingness matters.** Recovery can fail if too many low-frequency modes are missing or if missing coefficients cluster in a difficult range. The authors assume the important low modes are typically retained. 4. **Comparisons are selective.** SGSM is omitted in the high-dimensional experiments because it is deemed infeasible; PINNs and DRMs are omitted from the main middle-dimensional SGSM comparison because they perform poorly there. Those choices are motivated, but mean the headline is assembled from different benchmark regimes. 5. **Scope beyond tested PDEs.** Nonlinear evidence is limited to a 2-D Navier–Stokes coefficient-masking example. General nonlinear, complex-geometry, nonsmooth, or dense-spectrum problems need further study. 6. **Theoretical scope.** The basis-embedding statement depends on rank and approximation assumptions; the paper does not establish a broad, dimension-independent end-to-end guarantee for general PDE solutions. ## Takeaway Modified SINNs are a thoughtful attempt to make spectral representations learnable in dimensions where fully resolved spectral grids are impossible. Their empirical advantage comes from putting known harmonic structure—frequency-dependent decay and basis geometry—directly into a neural model. The approach is promising for smooth, sparse-spectrum high-dimensional PDEs and for repairing incomplete spectral solutions, but its claims should be interpreted within that structured regime rather than as a universal solution to high-dimensional PDEs. ## Source This summary was prepared from the 31-page PDF of arXiv:2607.13566 (v1, 15 July 2026), including the main text and appendices. > **Note:** This text was AI-generated.