Policy Compatible Skill Incremental Learning via Lazy Learning Interface
This addresses a specific bottleneck in hierarchical reinforcement learning for embodied agents, enabling more efficient and reusable skill acquisition, though it appears incremental as it builds on existing SIL methods.
The paper tackles the problem of skill-policy incompatibility in Skill Incremental Learning, where evolving skills disrupt existing policies, and proposes SIL-C, a framework that ensures compatibility, allowing skill improvements to enhance downstream policy performance without re-training.
Skill Incremental Learning (SIL) is the process by which an embodied agent expands and refines its skill set over time by leveraging experience gained through interaction with its environment or by the integration of additional data. SIL facilitates efficient acquisition of hierarchical policies grounded in reusable skills for downstream tasks. However, as the skill repertoire evolves, it can disrupt compatibility with existing skill-based policies, limiting their reusability and generalization. In this work, we propose SIL-C, a novel framework that ensures skill-policy compatibility, allowing improvements in incrementally learned skills to enhance the performance of downstream policies without requiring policy re-training or structural adaptation. SIL-C employs a bilateral lazy learning-based mapping technique to dynamically align the subtask space referenced by policies with the skill space decoded into agent behaviors. This enables each subtask, derived from the policy's decomposition of a complex task, to be executed by selecting an appropriate skill based on trajectory distribution similarity. We evaluate SIL-C across diverse SIL scenarios and demonstrate that it maintains compatibility between evolving skills and downstream policies while ensuring efficiency throughout the learning process.