Emergence of Symbols in Neural Networks for Semantic Understanding and Communication
This addresses the challenge of bridging connectionist and symbolic AI for more capable systems, though it appears incremental as it builds on existing approaches without claiming broad SOTA.
The paper tackles the problem of neural networks lacking the ability to generate symbols for higher cognitive functions like communication and reasoning, proposing SEA-net, which creates symbols that dynamically configure the network to perform tasks and capture compositional semantics, enabling new functions through symbolic manipulation or communication.
The capacity to generate meaningful symbols and effectively employ them for advanced cognitive processes, such as communication, reasoning, and planning, constitutes a fundamental and distinctive aspect of human intelligence. Existing deep neural networks still notably lag human capabilities in terms of generating symbols for higher cognitive functions. Here, we propose a solution (symbol emergence artificial network (SEA-net)) to endow neural networks with the ability to create symbols, understand semantics, and achieve communication. SEA-net generates symbols that dynamically configure the network to perform specific tasks. These symbols capture compositional semantic information that allows the system to acquire new functions purely by symbolic manipulation or communication. In addition, these self-generated symbols exhibit an intrinsic structure resembling that of natural language, suggesting a common framework underlying the generation and understanding of symbols in both human brains and artificial neural networks. We believe that the proposed framework will be instrumental in producing more capable systems that can synergize the strengths of connectionist and symbolic approaches for artificial intelligence (AI).