Multiclass Graph-Based Large Margin Classifiers: Unified Approach for Support Vectors and Neural Networks
This work addresses classification problems in machine learning by providing a unified geometric approach, though it is incremental as it builds on existing graph-based methods.
The paper tackles multiclass classification by extending Gabriel graph-based classifiers to neural networks, introducing smoother activation functions and structural support vector-centered neurons to improve margins and classification contours. Experimental results show the method outperforms previous graph-based classifiers and is statistically equivalent to tree-based models.
While large margin classifiers are originally an outcome of an optimization framework, support vectors (SVs) can be obtained from geometric approaches. This article presents advances in the use of Gabriel graphs (GGs) in binary and multiclass classification problems. For Chipclass, a hyperparameter-less and optimization-less GG-based binary classifier, we discuss how activation functions and support edge (SE)-centered neurons affect the classification, proposing smoother functions and structural SV (SSV)-centered neurons to achieve margins with low probabilities and smoother classification contours. We extend the neural network architecture, which can be trained with backpropagation with a softmax function and a cross-entropy loss, or by solving a system of linear equations. A new subgraph-/distance-based membership function for graph regularization is also proposed, along with a new GG recomputation algorithm that is less computationally expensive than the standard approach. Experimental results with the Friedman test show that our method was better than previous GG-based classifiers and statistically equivalent to tree-based models.