Adaptive Human-AI Coordination via Hierarchical Action Disentanglement
For human-AI collaboration, IAD addresses the problem of agents collapsing to single behaviors or learning poorly aligned skills, enabling more robust adaptation to diverse partners.
IAD introduces a deep hierarchical RL framework with an intrinsic reward for action disentanglement, enabling adaptive human-AI coordination. It outperforms baselines in Overcooked-AI across simulated and real human partners.
Human-AI collaboration requires agents that can adapt to diverse partner behaviors and skill levels while remaining robust to unseen partners. Existing methods often collapse to a single dominant behavior or learn poorly aligned skills, limiting effective coordination. We propose Intrinsic Action Disentanglement (IAD), a deep hierarchical reinforcement learning (DHRL) framework that learns distinct, partner-aware low-level action sequences conditioned on high-level latent skills. IAD introduces an intrinsic reward that explicitly encourages disentangled action distributions of the agent's low-level policy across skills, yielding an interpretable mapping between high-level decisions and partner-specific behavioral responses. By capturing temporally extended interaction patterns, IAD enables flexible adaptation to heterogeneous partner dynamics under distributional shift. We evaluate IAD in the Overcooked-AI domain across multiple layouts and diverse partner settings, including unseen simulated partners, a human-proxy model trained on human-human gameplay, and real human partners. Results show that IAD consistently outperforms strong baselines and achieves more reliable, adaptive coordination across all settings.