Challenges in Binary Classification
This work addresses the challenge of optimal classifier design in machine learning, but it is incremental as it builds on existing SVM methods without introducing a new solution.
The paper tackles the problem of finding an optimal binary classifier by framing it as a variational problem based on maximizing the Euclidean distance between classes, showing that SVM is a special case for linear classification but noting limitations for nonlinear cases, with no concrete results or numbers provided.
Binary Classification plays an important role in machine learning. For linear classification, SVM is the optimal binary classification method. For nonlinear classification, the SVM algorithm needs to complete the classification task by using the kernel function. Although the SVM algorithm with kernel function is very effective, the selection of kernel function is empirical, which means that the kernel function may not be optimal. Therefore, it is worth studying how to obtain an optimal binary classifier. In this paper, the problem of finding the optimal binary classifier is considered as a variational problem. We design the objective function of this variational problem through the max-min problem of the (Euclidean) distance between two classes. For linear classification, it can be deduced that SVM is a special case of this variational problem framework. For Euclidean distance, it is proved that the proposed variational problem has some limitations for nonlinear classification. Therefore, how to design a more appropriate objective function to find the optimal binary classifier is still an open problem. Further, it's discussed some challenges and problems in finding the optimal classifier.