Compositional Concept Generalization with Variational Quantum Circuits
This addresses the lack of compositional generalization in vision-language models, which is a key cognitive ability, but the approach is incremental as it builds on existing quantum and classical methods.
The paper tackled the problem of compositional generalization in AI by using variational quantum circuits to learn representations from compositional tensor-based models on an image captioning task, achieving proof-of-concept results with noisy multi-hot encodings and outperforming classical models on CLIP vectors.
Compositional generalization is a key facet of human cognition, but lacking in current AI tools such as vision-language models. Previous work examined whether a compositional tensor-based sentence semantics can overcome the challenge, but led to negative results. We conjecture that the increased training efficiency of quantum models will improve performance in these tasks. We interpret the representations of compositional tensor-based models in Hilbert spaces and train Variational Quantum Circuits to learn these representations on an image captioning task requiring compositional generalization. We used two image encoding techniques: a multi-hot encoding (MHE) on binary image vectors and an angle/amplitude encoding on image vectors taken from the vision-language model CLIP. We achieve good proof-of-concept results using noisy MHE encodings. Performance on CLIP image vectors was more mixed, but still outperformed classical compositional models.