Hierarchical Models: Intrinsic Separability in High Dimensions
This addresses the problem of inconsistent interpretations of high-dimensional data for machine learning researchers, offering a foundational reconciliation.
The paper tackles the inconsistent patterns in high-dimensional data by proposing an intrinsically hierarchical generative process, which reconciles various theories and results, and demonstrates that this leads to qualitative and quantitative improvements in open-set learning performance.
It has long been noticed that high dimension data exhibits strange patterns. This has been variously interpreted as either a "blessing" or a "curse", causing uncomfortable inconsistencies in the literature. We propose that these patterns arise from an intrinsically hierarchical generative process. Modeling the process creates a web of constraints that reconcile many different theories and results. The model also implies high dimensional data posses an innate separability that can be exploited for machine learning. We demonstrate how this permits the open-set learning problem to be defined mathematically, leading to qualitative and quantitative improvements in performance.