Distance Assessment and Hypothesis Testing of High-Dimensional Samples using Variational Autoencoders
This provides a tool for data exploration in early phases of machine learning projects, but it is incremental as it builds on existing variational autoencoder methods.
The paper tackles the problem of assessing whether two high-dimensional datasets come from the same distribution by introducing a distance measurement using variational autoencoders, augmented with a permutation hypothesis test, and shows it can quantify discrepancies between categories like images for data exploration.
Given two distinct datasets, an important question is if they have arisen from the the same data generating function or alternatively how their data generating functions diverge from one another. In this paper, we introduce an approach for measuring the distance between two datasets with high dimensionality using variational autoencoders. This approach is augmented by a permutation hypothesis test in order to check the hypothesis that the data generating distributions are the same within a significance level. We evaluate both the distance measurement and hypothesis testing approaches on generated and on public datasets. According to the results the proposed approach can be used for data exploration (e.g. by quantifying the discrepancy/separability between categories of images), which can be particularly useful in the early phases of the pipeline of most machine learning projects.