A Game Interface to Study Semantic Grounding in Text-Based Models
This work addresses a central problem in NLP regarding semantic grounding for researchers, but it is incremental as it focuses on early data collection and testing.
The authors tackled the question of whether language models can learn grounded representations from text alone by proposing an experimental test: if words with different meanings cannot be distinguished distributionally, grounding is impossible for text-based models. They developed an online game to collect human judgments on word pair similarity in five languages and reported early results from data collection.
Can language models learn grounded representations from text distribution alone? This question is both central and recurrent in natural language processing; authors generally agree that grounding requires more than textual distribution. We propose to experimentally test this claim: if any two words have different meanings and yet cannot be distinguished from distribution alone, then grounding is out of the reach of text-based models. To that end, we present early work on an online game for the collection of human judgments on the distributional similarity of word pairs in five languages. We further report early results of our data collection campaign.