Feature Entanglement-based Quantum Multimodal Fusion Neural Network
This addresses the problem of balancing accuracy, interpretability, and complexity in multimodal learning for researchers, though it appears incremental as it builds on quantum computation frameworks.
The paper tackles the accuracy-interpretability-complexity trade-off in multimodal learning by proposing a feature entanglement-based quantum multimodal fusion neural network, achieving classification accuracy comparable to classical networks with dozens of times fewer parameters in simulations on multimodal image datasets.
Multimodal learning aims to enhance perceptual and decision-making capabilities by integrating information from diverse sources. However, classical deep learning approaches face a critical trade-off between the high accuracy of black-box feature-level fusion and the interpretability of less outstanding decision-level fusion, alongside the challenges of parameter explosion and complexity. This paper discusses the accuracy-interpretablity-complexity dilemma under the quantum computation framework and propose a feature entanglement-based quantum multimodal fusion neural network. The model is composed of three core components: a classical feed-forward module for unimodal processing, an interpretable quantum fusion block, and a quantum convolutional neural network (QCNN) for deep feature extraction. By leveraging the strong expressive power of quantum, we have reduced the complexity of multimodal fusion and post-processing to linear, and the fusion process also possesses the interpretability of decision-level fusion. The simulation results demonstrate that our model achieves classification accuracy comparable to classical networks with dozens of times of parameters, exhibiting notable stability and performance across multimodal image datasets.