Learning to be Reproducible: Custom Loss Design for Robust Neural Networks
This addresses the need for more reliable and reproducible neural networks, particularly in applications requiring consistent outcomes, though it is incremental as it builds on existing training methodologies.
The paper tackles the problem of inconsistent and variable performance of deep learning models across training runs by proposing a Custom Loss Function (CLF) that balances accuracy with stability, resulting in improved training robustness without sacrificing predictive performance.
To enhance the reproducibility and reliability of deep learning models, we address a critical gap in current training methodologies: the lack of mechanisms that ensure consistent and robust performance across runs. Our empirical analysis reveals that even under controlled initialization and training conditions, the accuracy of the model can exhibit significant variability. To address this issue, we propose a Custom Loss Function (CLF) that reduces the sensitivity of training outcomes to stochastic factors such as weight initialization and data shuffling. By fine-tuning its parameters, CLF explicitly balances predictive accuracy with training stability, leading to more consistent and reliable model performance. Extensive experiments across diverse architectures for both image classification and time series forecasting demonstrate that our approach significantly improves training robustness without sacrificing predictive performance. These results establish CLF as an effective and efficient strategy for developing more stable, reliable and trustworthy neural networks.