A theory of understanding for artificial intelligence: composability, catalysts, and learning
This work addresses the foundational challenge of defining understanding for AI systems, which is crucial for advancing towards general intelligence, but it is incremental as it builds on existing concepts like composability without introducing new empirical methods.
The paper tackles the problem of defining and analyzing understanding in AI by proposing a composability-based framework, which characterizes understanding as the ability to process inputs into outputs from a verifier's perspective and introduces the concept of catalysts to enhance output quality. It suggests that models like language models, which can generate outputs acting as their own catalysts, may help overcome limitations in AI understanding.
Understanding is a crucial yet elusive concept in artificial intelligence (AI). This work proposes a framework for analyzing understanding based on the notion of composability. Given any subject (e.g., a person or an AI), we suggest characterizing its understanding of an object in terms of its ability to process (compose) relevant inputs into satisfactory outputs from the perspective of a verifier. This highly universal framework can readily apply to non-human subjects, such as AIs, non-human animals, and institutions. Further, we propose methods for analyzing the inputs that enhance output quality in compositions, which we call catalysts. We show how the structure of a subject can be revealed by analyzing its components that act as catalysts and argue that a subject's learning ability can be regarded as its ability to compose inputs into its inner catalysts. Finally we examine the importance of learning ability for AIs to attain general intelligence. Our analysis indicates that models capable of generating outputs that can function as their own catalysts, such as language models, establish a foundation for potentially overcoming existing limitations in AI understanding.