Multimodal Datasets with Controllable Mutual Information
This provides a novel testbed for researchers in machine learning to assess methods in multimodal learning and information theory, though it is incremental as it builds on existing generative and causal models.
The authors tackled the problem of evaluating mutual information estimators and multimodal self-supervised learning by introducing a framework to generate multimodal datasets with controllable and calculable mutual information, enabling systematic benchmarking.
We introduce a framework for generating highly multimodal datasets with explicitly calculable mutual information between modalities. This enables the construction of benchmark datasets that provide a novel testbed for systematic studies of mutual information estimators and multimodal self-supervised learning techniques. Our framework constructs realistic datasets with known mutual information using a flow-based generative model and a structured causal framework for generating correlated latent variables.