Equivalence of Learning Algorithms
This work addresses a foundational issue for researchers in machine learning by providing a framework to compare and transfer algorithm properties, though it appears incremental as it builds on existing regularization methods.
The paper tackles the problem of defining equivalence between machine learning algorithms, introducing weak and strong equivalence concepts to enable transfer of learning properties, and illustrates this by analyzing the relation between kernel ridge regression and m-power regularized least squares regression.
The purpose of this paper is to introduce a concept of equivalence between machine learning algorithms. We define two notions of algorithmic equivalence, namely, weak and strong equivalence. These notions are of paramount importance for identifying when learning prop erties from one learning algorithm can be transferred to another. Using regularized kernel machines as a case study, we illustrate the importance of the introduced equivalence concept by analyzing the relation between kernel ridge regression (KRR) and m-power regularized least squares regression (M-RLSR) algorithms.