Goal-Directedness is in the Eye of the Beholder
This work addresses a foundational issue in AI and cognitive science for researchers studying agent behavior, but it is incremental as it critiques existing approaches without presenting new empirical results.
The paper tackles the problem of attributing goals to complex agents by analyzing behavioral and mechanistic approaches, concluding that goal-directedness cannot be objectively measured. It proposes new directions for modeling goal-directedness as emergent in dynamic, multi-agent systems.
Our ability to predict the behavior of complex agents turns on the attribution of goals. Probing for goal-directed behavior comes in two flavors: Behavioral and mechanistic. The former proposes that goal-directedness can be estimated through behavioral observation, whereas the latter attempts to probe for goals in internal model states. We work through the assumptions behind both approaches, identifying technical and conceptual problems that arise from formalizing goals in agent systems. We arrive at the perhaps surprising position that goal-directedness cannot be measured objectively. We outline new directions for modeling goal-directedness as an emergent property of dynamic, multi-agent systems.