Bootstrapping Concept Formation in Small Neural Networks
This work addresses the fundamental challenge of concept formation in neural systems, potentially applicable to understanding pre-linguistic animals, but it is incremental as it builds on associative learning principles without demonstrating broad SOTA impact.
The paper tackles the problem of how neural systems form concepts for reasoning by proposing a model where a small neural network agent uses realistic learning rules and environmental feedback to develop relational representations from statistical regularities, resulting in the formation of primordial concepts that replace reflexive actions with concept-driven behavior.
The question how neural systems (of humans) can perform reasoning is still far from being solved. We posit that the process of forming Concepts is a fundamental step required for this. We argue that, first, Concepts are formed as closed representations, which are then consolidated by relating them to each other. Here we present a model system (agent) with a small neural network that uses realistic learning rules and receives only feedback from the environment in which the agent performs virtual actions. First, the actions of the agent are reflexive. In the process of learning, statistical regularities in the input lead to the formation of neuronal pools representing relations between the entities observed by the agent from its artificial world. This information then influences the behavior of the agent via feedback connections replacing the initial reflex by an action driven by these relational representations. We hypothesize that the neuronal pools representing relational information can be considered as primordial Concepts, which may in a similar way be present in some pre-linguistic animals, too. This system provides formal grounds for further discussions on what could be understood as a Concept and shows that associative learning is enough to develop concept-like structures.