Abstraction in decision-makers with limited information processing capabilities
This work addresses a foundational issue in AI and cognitive science by explaining abstraction formation, though it appears incremental as it builds on existing theories without introducing a new method.
The paper tackles the problem of how abstractions emerge in decision-making systems with limited information processing capabilities by linking the free-energy framework to rate-distortion theory, demonstrating that abstractions arise naturally from these constraints.
A distinctive property of human and animal intelligence is the ability to form abstractions by neglecting irrelevant information which allows to separate structure from noise. From an information theoretic point of view abstractions are desirable because they allow for very efficient information processing. In artificial systems abstractions are often implemented through computationally costly formations of groups or clusters. In this work we establish the relation between the free-energy framework for decision making and rate-distortion theory and demonstrate how the application of rate-distortion for decision-making leads to the emergence of abstractions. We argue that abstractions are induced due to a limit in information processing capacity.