Unitary-Group Invariant Kernels and Features from Transformed Unlabeled Data
This work addresses a practical challenge in machine learning for applications where labeled transformed data is unavailable, offering a theoretically grounded solution with potential domain-specific impact.
The paper tackles the problem of learning representations invariant to unitary group transformations without requiring labeled transformed data or observing all transformations, proposing a new invariant kernel SVM and feature extraction method with theoretical guarantees and experimental validation on UCI ML datasets.
The study of representations invariant to common transformations of the data is important to learning. Most techniques have focused on local approximate invariance implemented within expensive optimization frameworks lacking explicit theoretical guarantees. In this paper, we study kernels that are invariant to the unitary group while having theoretical guarantees in addressing practical issues such as (1) unavailability of transformed versions of labelled data and (2) not observing all transformations. We present a theoretically motivated alternate approach to the invariant kernel SVM. Unlike previous approaches to the invariant SVM, the proposed formulation solves both issues mentioned. We also present a kernel extension of a recent technique to extract linear unitary-group invariant features addressing both issues and extend some guarantees regarding invariance and stability. We present experiments on the UCI ML datasets to illustrate and validate our methods.