Modeling the Formation of Social Conventions from Embodied Real-Time Interactions
This work addresses the formation of social conventions for researchers in AI and cognitive science, though it is incremental as it builds on existing theories like Distributed Adaptive Control.
The authors tackled the problem of how real-time control and learning influence social convention formation by proposing a Control-based Reinforcement Learning (CRL) model that matches human behavioral data in a social decision-making game, achieving human-level performance in efficiency and fairness metrics.
What is the role of real-time control and learning in the formation of social conventions? To answer this question, we propose a computational model that matches human behavioral data in a social decision-making game that was analyzed both in discrete-time and continuous-time setups. Furthermore, unlike previous approaches, our model takes into account the role of sensorimotor control loops in embodied decision-making scenarios. For this purpose, we introduce the Control-based Reinforcement Learning (CRL) model. CRL is grounded in the Distributed Adaptive Control (DAC) theory of mind and brain, where low-level sensorimotor control is modulated through perceptual and behavioral learning in a layered structure. CRL follows these principles by implementing a feedback control loop handling the agent's reactive behaviors (pre-wired reflexes), along with an adaptive layer that uses reinforcement learning to maximize long-term reward. We test our model in a multi-agent game-theoretic task in which coordination must be achieved to find an optimal solution. We show that CRL is able to reach human-level performance on standard game-theoretic metrics such as efficiency in acquiring rewards and fairness in reward distribution.