Modularity-Free Conflict-Averse Training for Generalized PINNs
For researchers using PINNs, this work addresses a previously unrecognized failure mode in overparameterized networks, offering a method to maintain training robustness and accuracy.
The paper identifies a capacity-induced failure mode in PINNs where overparameterized networks develop functional modularity that hinders convergence, and proposes ModSync, which integrates structural optimization into conflict-averse training to prevent this, achieving state-of-the-art accuracy across diverse PDE benchmarks.
Physics-informed neural networks (PINNs) have become a powerful framework for solving PDEs by embedding physical laws into differentiable objectives. Despite their advances, training PINNs remains fragile: recent conflict-averse optimization schemes alleviate gradient interference between residual and boundary losses, but we show that their effectiveness deteriorates as model capacity increases. In this paper, we identify a capacity-induced failure mode, where overparameterized networks undergo functional modularity, self-partitioning into task-exclusive modules that suppress cross-objective interaction and hinder convergence toward Pareto-stationary points. To address this issue, we propose a novel framework, Modular-Sparsity Synchronization (ModSync), which integrates structural optimization into conflict-averse training by penalizing task-exclusive connections while preserving interaction-promoting pathways. Extensive experiments across diverse PDE benchmarks demonstrate that ModSync consistently prevents capacity-driven failures, sustains robust cross-objective coupling, and achieves state-of-the-art accuracy. Codes are available at \url{https://github.com/heejokong/ModSync}.