An Information-theoretic On-line Learning Principle for Specialization in Hierarchical Decision-Making Systems
This work addresses the challenge of scalable and efficient decision-making in hierarchical systems for AI and machine learning applications, though it appears incremental as it builds on existing bounded rationality frameworks.
The paper tackles the problem of enabling resource-limited decision-makers to collaboratively solve complex tasks beyond individual capabilities by proposing an information-theoretic principle for specialization and division of labor. It introduces an on-line learning rule that partitions the problem space for specialized linear policies, demonstrating applicability in classification, regression, reinforcement learning, and adaptive control.
Information-theoretic bounded rationality describes utility-optimizing decision-makers whose limited information-processing capabilities are formalized by information constraints. One of the consequences of bounded rationality is that resource-limited decision-makers can join together to solve decision-making problems that are beyond the capabilities of each individual. Here, we study an information-theoretic principle that drives division of labor and specialization when decision-makers with information constraints are joined together. We devise an on-line learning rule of this principle that learns a partitioning of the problem space such that it can be solved by specialized linear policies. We demonstrate the approach for decision-making problems whose complexity exceeds the capabilities of individual decision-makers, but can be solved by combining the decision-makers optimally. The strength of the model is that it is abstract and principled, yet has direct applications in classification, regression, reinforcement learning and adaptive control.