Feature Space Topology Control via Hopkins Loss
This work addresses the need for modifying feature space topology in applications such as dimensionality reduction and robustness, but it is incremental as it builds on existing topology-related methods.
The paper tackled the problem of controlling feature space topology in machine learning by introducing Hopkins loss, a novel loss function based on the Hopkins statistic, and found that it modifies feature topology with only a small impact on classification performance in tasks like classification and dimensionality reduction.
Feature space topology refers to the organization of samples within the feature space. Modifying this topology can be beneficial in machine learning applications, including dimensionality reduction, generative modeling, transfer learning, and robustness to adversarial attacks. This paper introduces a novel loss function, Hopkins loss, which leverages the Hopkins statistic to enforce a desired feature space topology, which is in contrast to existing topology-related methods that aim to preserve input feature topology. We evaluate the effectiveness of Hopkins loss on speech, text, and image data in two scenarios: classification and dimensionality reduction using nonlinear bottleneck autoencoders. Our experiments show that integrating Hopkins loss into classification or dimensionality reduction has only a small impact on classification performance while providing the benefit of modifying feature topology.