Understanding understanding: a renormalization group inspired model of (artificial) intelligence
This work addresses foundational issues in AI theory, offering a novel perspective on understanding that could influence how intelligent systems are designed and evaluated, though it appears incremental in its application of existing physical concepts.
The paper tackles the problem of defining 'understanding' in AI by proposing a mathematical model based on renormalization group theory, treating transformations as reorganizations of information rather than losses, and introduces a relevance measure for lossy compression.
This paper is about the meaning of understanding in scientific and in artificial intelligent systems. We give a mathematical definition of the understanding, where, contrary to the common wisdom, we define the probability space on the input set, and we treat the transformation made by an intelligent actor not as a loss of information, but instead a reorganization of the information in the framework of a new coordinate system. We introduce, following the ideas of physical renormalization group, the notions of relevant and irrelevant parameters, and discuss, how the different AI tasks can be interpreted along these concepts, and how the process of learning can be described. We show, how scientific understanding fits into this framework, and demonstrate, what is the difference between a scientific task and pattern recognition. We also introduce a measure of relevance, which is useful for performing lossy compression.