Compositional properties of emergent languages in deep learning
This work addresses the problem of evaluating language compositionality in AI systems for researchers, but it is incremental as it critiques existing claims without introducing a new method.
The paper analyzes emergent languages in multi-agent deep learning systems, finding that these languages often lack compositionality and fail to generalize to out-of-distribution examples, with no concrete numerical results provided.
Recent findings in multi-agent deep learning systems point towards the emergence of compositional languages. These claims are often made without exact analysis or testing of the language. In this work, we analyze the emergent language resulting from two different cooperative multi-agent game with more exact measures for compositionality. Our findings suggest that solutions found by deep learning models are often lacking the ability to reason on an abstract level therefore failing to generalize the learned knowledge to out of the training distribution examples. Strategies for testing compositional capacities and emergence of human-level concepts are discussed.