Visual Theory of Mind Enables the Invention of Proto-Writing
This work addresses the emergence of symbolic writing systems, a foundational aspect of human communication, but it is incremental as it builds on existing computational models of communication.
The study tackled the problem of how proto-writing systems emerge by developing a multi-agent reinforcement learning testbed called the Signification Game, enabling agents to use visual theory of mind to communicate actions with pictographs, and it provided insights into the cognitive and cultural processes involved.
Symbolic writing systems are graphical semiotic codes that are ubiquitous in modern society but are otherwise absent in the animal kingdom. Anthropological evidence suggests that the earliest forms of some writing systems originally consisted of iconic pictographs, which signify their referent via visual resemblance. While previous studies have examined the emergence and, separately, the evolution of pictographic systems through a computational lens, most employ non-naturalistic methodologies that make it difficult to draw clear analogies to human and animal cognition. We develop a multi-agent reinforcement learning testbed for emergent communication called a Signification Game, and formulate a model of inferential communication that enables agents to leverage visual theory of mind to communicate actions using pictographs. Our model, which is situated within a broader formalism for animal communication, sheds light on the cognitive and cultural processes underlying the emergence of proto-writing.