Modal Uncertainty Estimation via Discrete Latent Representation
This addresses the need for models to propose multiple plausible outputs with reliable uncertainty measures, which is crucial for real-world applications where solutions are not unique, though it appears incremental as it builds on existing conditional generative models.
The paper tackles the problem of modeling one-to-many mappings in machine learning by introducing a framework that uses discrete latent variables to represent different plausible solutions, resulting in significantly more accurate uncertainty estimation compared to state-of-the-art methods.
Many important problems in the real world don't have unique solutions. It is thus important for machine learning models to be capable of proposing different plausible solutions with meaningful probability measures. In this work we introduce such a deep learning framework that learns the one-to-many mappings between the inputs and outputs, together with faithful uncertainty measures. We call our framework {\it modal uncertainty estimation} since we model the one-to-many mappings to be generated through a set of discrete latent variables, each representing a latent mode hypothesis that explains the corresponding type of input-output relationship. The discrete nature of the latent representations thus allows us to estimate for any input the conditional probability distribution of the outputs very effectively. Both the discrete latent space and its uncertainty estimation are jointly learned during training. We motivate our use of discrete latent space through the multi-modal posterior collapse problem in current conditional generative models, then develop the theoretical background, and extensively validate our method on both synthetic and realistic tasks. Our framework demonstrates significantly more accurate uncertainty estimation than the current state-of-the-art methods, and is informative and convenient for practical use.