DFReg: A Physics-Inspired Framework for Global Weight Distribution Regularization in Neural Networks
This work addresses the challenge of regularization in deep learning for researchers and practitioners, offering a novel approach that is incremental compared to existing methods like Dropout or L2 decay.
The authors tackled the problem of improving neural network training by introducing DFReg, a physics-inspired regularization method that applies a functional penalty to encourage smooth and diverse global weight distributions, resulting in enhanced model performance without architectural modifications.
We introduce DFReg, a physics-inspired regularization method for deep neural networks that operates on the global distribution of weights. Drawing from Density Functional Theory (DFT), DFReg applies a functional penalty to encourage smooth, diverse, and well-distributed weight configurations. Unlike traditional techniques such as Dropout or L2 decay, DFReg imposes global structural regularity without architectural changes or stochastic perturbations.