Toward Compositional Generalization in Object-Oriented World Modeling
This work addresses a fundamental challenge in AI for improving learning efficiency in structured environments, though it appears incremental with a focus on specific methods.
The paper tackles compositional generalization in reinforcement learning for object-oriented environments by formalizing the problem algebraically and introducing a differentiable approach, HOWM, which achieves soft but efficient generalization.
Compositional generalization is a critical ability in learning and decision-making. We focus on the setting of reinforcement learning in object-oriented environments to study compositional generalization in world modeling. We (1) formalize the compositional generalization problem with an algebraic approach and (2) study how a world model can achieve that. We introduce a conceptual environment, Object Library, and two instances, and deploy a principled pipeline to measure the generalization ability. Motivated by the formulation, we analyze several methods with exact or no compositional generalization ability using our framework, and design a differentiable approach, Homomorphic Object-oriented World Model (HOWM), that achieves soft but more efficient compositional generalization.