CLIP also Understands Text: Prompting CLIP for Phrase Understanding
This work addresses text understanding for NLP applications, but it is incremental as it applies an existing model to a new task.
The paper tackled the problem of text understanding using CLIP's text encoder, showing it outperforms BERT on phrase understanding tasks with proper prompting, achieving strong results across datasets in entity clustering and expansion.
Contrastive Language-Image Pretraining (CLIP) efficiently learns visual concepts by pre-training with natural language supervision. CLIP and its visual encoder have been explored on various vision and language tasks and achieve strong zero-shot or transfer learning performance. However, the application of its text encoder solely for text understanding has been less explored. In this paper, we find that the text encoder of CLIP actually demonstrates strong ability for phrase understanding, and can even significantly outperform popular language models such as BERT with a properly designed prompt. Extensive experiments validate the effectiveness of our method across different datasets and domains on entity clustering and entity set expansion tasks.