Machine Generalization and Human Categorization: An Information-Theoretic View
This work addresses the challenge of making AI systems more interpretable and aligned with human cognitive processes, which is incremental as it applies existing information theory to a known bottleneck in human-AI interaction.
The paper tackled the problem of designing intelligent systems that can explain their reasoning and provide generalizations that humans find reasonable by incorporating psychological data on human categorization. The result showed that an information-theoretic measure of category value predicts experimental outcomes better than standard alternatives, suggesting such approaches are useful for human-compatible machine generalization.
In designing an intelligent system that must be able to explain its reasoning to a human user, or to provide generalizations that the human user finds reasonable, it may be useful to take into consideration psychological data on what types of concepts and categories people naturally use. The psychological literature on concept learning and categorization provides strong evidence that certain categories are more easily learned, recalled, and recognized than others. We show here how a measure of the informational value of a category predicts the results of several important categorization experiments better than standard alternative explanations. This suggests that information-based approaches to machine generalization may prove particularly useful and natural for human users of the systems.