RoBoSS: A Robust, Bounded, Sparse, and Smooth Loss Function for Supervised Learning
This addresses the need for more robust loss functions in machine learning, particularly for handling outliers and noise, but appears incremental as it builds on existing SVM methods.
The paper tackles the problem of traditional loss functions struggling with outlier-prone and high-dimensional data in supervised learning by proposing a novel robust, bounded, sparse, and smooth (RoBoSS) loss function, integrated into an SVM framework, and shows superior generalization and training efficiency on 88 benchmark datasets and medical datasets.
In the domain of machine learning, the significance of the loss function is paramount, especially in supervised learning tasks. It serves as a fundamental pillar that profoundly influences the behavior and efficacy of supervised learning algorithms. Traditional loss functions, though widely used, often struggle to handle outlier-prone and high-dimensional data, resulting in suboptimal outcomes and slow convergence during training. In this paper, we address the aforementioned constraints by proposing a novel robust, bounded, sparse, and smooth (RoBoSS) loss function for supervised learning. Further, we incorporate the RoBoSS loss within the framework of support vector machine (SVM) and introduce a new robust algorithm named $\mathcal{L}_{RoBoSS}$-SVM. For the theoretical analysis, the classification-calibrated property and generalization ability are also presented. These investigations are crucial for gaining deeper insights into the robustness of the RoBoSS loss function in classification problems and its potential to generalize well to unseen data. To validate the potency of the proposed $\mathcal{L}_{RoBoSS}$-SVM, we assess it on $88$ benchmark datasets from KEEL and UCI repositories. Further, to rigorously evaluate its performance in challenging scenarios, we conducted an assessment using datasets intentionally infused with outliers and label noise. Additionally, to exemplify the effectiveness of $\mathcal{L}_{RoBoSS}$-SVM within the biomedical domain, we evaluated it on two medical datasets: the electroencephalogram (EEG) signal dataset and the breast cancer (BreaKHis) dataset. The numerical results substantiate the superiority of the proposed $\mathcal{L}_{RoBoSS}$-SVM model, both in terms of its remarkable generalization performance and its efficiency in training time.