Hierarchical Mutual Information Analysis: Towards Multi-view Clustering in The Wild
This addresses a practical issue in computer vision applications where multi-view data often suffer from view missing and misalignment, though it appears incremental as it builds on existing MVC techniques.
The paper tackles the problem of multi-view clustering with missing and unaligned data by proposing a deep framework that fuses data recovery and alignment hierarchically to maximize mutual information and ensure latent space consistency, achieving significant performance improvements over state-of-the-art methods on public datasets.
Multi-view clustering (MVC) can explore common semantics from unsupervised views generated by different sources, and thus has been extensively used in applications of practical computer vision. Due to the spatio-temporal asynchronism, multi-view data often suffer from view missing and are unaligned in real-world applications, which makes it difficult to learn consistent representations. To address the above issues, this work proposes a deep MVC framework where data recovery and alignment are fused in a hierarchically consistent way to maximize the mutual information among different views and ensure the consistency of their latent spaces. More specifically, we first leverage dual prediction to fill in missing views while achieving the instance-level alignment, and then take the contrastive reconstruction to achieve the class-level alignment. To the best of our knowledge, this could be the first successful attempt to handle the missing and unaligned data problem separately with different learning paradigms. Extensive experiments on public datasets demonstrate that our method significantly outperforms state-of-the-art methods on multi-view clustering even in the cases of view missing and unalignment.