A Study on Variants of Conventional, Fuzzy, and Nullspace-Based Independence Criteria for Improving Supervised and Unsupervised Learning
This work addresses the need for interpretable machine learning methods in both supervised and unsupervised learning, though it appears incremental as it builds on existing independence criteria.
The authors tackled the problem of designing unsupervised and supervised dimensionality reduction methods by proposing three new independence criteria, which outperformed baseline methods like tSNE, PCA, and VAE in terms of contrast, accuracy, and interpretability.
Unsupervised and supervised learning methods conventionally use kernels to capture nonlinearities inherent in data structure. However experts have to ensure their proposed nonlinearity maximizes variability and capture inherent diversity of data. We reviewed all independence criteria to design unsupervised learners. Then we proposed 3 independence criteria and used them to design unsupervised and supervised dimensionality reduction methods. We evaluated contrast, accuracy and interpretability of these methods in both linear and neural nonlinear settings. The results show that the methods have outperformed the baseline (tSNE, PCA, regularized LDA, VAE with (un)supervised learner and layer sharing) and opened a new line of interpretable machine learning (ML) for the researchers.