Topologically Densified Distributions
This addresses generalization issues in neural networks for vision tasks, but it is incremental as it builds on prior work for imposing topological constraints.
The paper tackles the problem of small sample-size learning with over-parameterized neural networks by imposing a topological constraint on internal representations, which provably leads to mass concentration around training instances and shows empirical evidence for better generalization across vision benchmarks.
We study regularization in the context of small sample-size learning with over-parameterized neural networks. Specifically, we shift focus from architectural properties, such as norms on the network weights, to properties of the internal representations before a linear classifier. Specifically, we impose a topological constraint on samples drawn from the probability measure induced in that space. This provably leads to mass concentration effects around the representations of training instances, i.e., a property beneficial for generalization. By leveraging previous work to impose topological constraints in a neural network setting, we provide empirical evidence (across various vision benchmarks) to support our claim for better generalization.