Designing a Robust, Bounded, and Smooth Loss Function for Improved Supervised Learning
This work addresses a fundamental issue in machine learning for practitioners dealing with noisy or outlier-prone data, though it appears incremental as it builds on existing loss function concepts.
The paper tackles the problem of traditional loss functions struggling with high-dimensional and outlier-sensitive datasets in supervised learning, resulting in reduced performance and slower convergence, by developing a robust, bounded, and smooth loss function (RoBoS-NN) that outperforms benchmark models in accuracy on real-world datasets with infused outliers.
The loss function is crucial to machine learning, especially in supervised learning frameworks. It is a fundamental component that controls the behavior and general efficacy of learning algorithms. However, despite their widespread use, traditional loss functions have significant drawbacks when dealing with high-dimensional and outlier-sensitive datasets, which frequently results in reduced performance and slower convergence during training. In this work, we develop a robust, bounded, and smooth (RoBoS-NN) loss function to resolve the aforementioned hindrances. The generalization ability of the loss function has also been theoretically analyzed to rigorously justify its robustness. Moreover, we implement RoboS-NN loss in the framework of a neural network (NN) to forecast time series and present a new robust algorithm named $\mathcal{L}_{\text{RoBoS}}$-NN. To assess the potential of $\mathcal{L}_{\text{RoBoS}}$-NN, we conduct experiments on multiple real-world datasets. In addition, we infuse outliers into data sets to evaluate the performance of $\mathcal{L}_{\text{RoBoS}}$-NN in more challenging scenarios. Numerical results show that $\mathcal{L}_{\text{RoBoS}}$-NN outperforms the other benchmark models in terms of accuracy measures.