Semi-supervised classification by reaching consensus among modalities
This addresses data scarcity in multi-modal learning for applications like banking and healthcare, though it appears incremental as an extension of existing Consensus Networks.
The paper tackles the problem of data sparsity in deep learning by extending Consensus Networks to Transductive Consensus Networks (TCNs) for semi-supervised classification, achieving performance that matches or exceeds benchmark algorithms with only 20 to 200 labeled samples on Bank Marketing and DementiaBank datasets.
Deep learning has demonstrated abilities to learn complex structures, but they can be restricted by available data. Recently, Consensus Networks (CNs) were proposed to alleviate data sparsity by utilizing features from multiple modalities, but they too have been limited by the size of labeled data. In this paper, we extend CN to Transductive Consensus Networks (TCNs), suitable for semi-supervised learning. In TCNs, different modalities of input are compressed into latent representations, which we encourage to become indistinguishable during iterative adversarial training. To understand TCNs two mechanisms, consensus and classification, we put forward its three variants in ablation studies on these mechanisms. To further investigate TCN models, we treat the latent representations as probability distributions and measure their similarities as the negative relative Jensen-Shannon divergences. We show that a consensus state beneficial for classification desires a stable but imperfect similarity between the representations. Overall, TCNs outperform or align with the best benchmark algorithms given 20 to 200 labeled samples on the Bank Marketing and the DementiaBank datasets.