Towards Visual Taxonomy Expansion
This work addresses the inability to generalize to unseen terms in taxonomy expansion, which is incremental as it builds on existing methods by adding visual semantics.
The paper tackles the taxonomy expansion problem by introducing visual features to address limitations of text-only methods, achieving an 8.75% accuracy improvement on a Chinese dataset and outperforming ChatGPT.
Taxonomy expansion task is essential in organizing the ever-increasing volume of new concepts into existing taxonomies. Most existing methods focus exclusively on using textual semantics, leading to an inability to generalize to unseen terms and the "Prototypical Hypernym Problem." In this paper, we propose Visual Taxonomy Expansion (VTE), introducing visual features into the taxonomy expansion task. We propose a textual hypernymy learning task and a visual prototype learning task to cluster textual and visual semantics. In addition to the tasks on respective modalities, we introduce a hyper-proto constraint that integrates textual and visual semantics to produce fine-grained visual semantics. Our method is evaluated on two datasets, where we obtain compelling results. Specifically, on the Chinese taxonomy dataset, our method significantly improves accuracy by 8.75 %. Additionally, our approach performs better than ChatGPT on the Chinese taxonomy dataset.