Bundle Networks: Fiber Bundles, Local Trivializations, and a Generative Approach to Exploring Many-to-one Maps
This addresses the need for better interpretability and exploration in models with many-to-one mappings, such as image recognition and time series forecasting, offering a foundational approach rather than an incremental improvement.
The paper tackles the problem of exploring and sampling from the fibers (sets of inputs mapping to the same output) in many-to-one maps in machine learning, showing that existing generative architectures are ill-suited and introducing Bundle Networks, a novel architecture based on fiber bundles and local trivializations, which enables natural investigation of fibers using state-of-the-art invertible components.
Many-to-one maps are ubiquitous in machine learning, from the image recognition model that assigns a multitude of distinct images to the concept of "cat" to the time series forecasting model which assigns a range of distinct time-series to a single scalar regression value. While the primary use of such models is naturally to associate correct output to each input, in many problems it is also useful to be able to explore, understand, and sample from a model's fibers, which are the set of input values $x$ such that $f(x) = y$, for fixed $y$ in the output space. In this paper we show that popular generative architectures are ill-suited to such tasks. Motivated by this we introduce a novel generative architecture, a Bundle Network, based on the concept of a fiber bundle from (differential) topology. BundleNets exploit the idea of a local trivialization wherein a space can be locally decomposed into a product space that cleanly encodes the many-to-one nature of the map. By enforcing this decomposition in BundleNets and by utilizing state-of-the-art invertible components, investigating a network's fibers becomes natural.