MRSD: Multi-Resolution Skill Discovery for HRL Agents
This work addresses a bottleneck in HRL for robotics and AI agents by enabling more versatile and efficient control, though it is incremental in building on existing skill discovery methods.
The paper tackles the limitation of single-skill learning in hierarchical reinforcement learning by proposing MRSD, a framework that learns multiple skill encoders at different temporal resolutions, which outperforms prior methods on DeepMind Control Suite tasks with faster convergence and higher final performance.
Hierarchical reinforcement learning (HRL) relies on abstract skills to solve long-horizon tasks efficiently. While existing skill discovery methods learns these skills automatically, they are limited to a single skill per task. In contrast, humans learn and use both fine-grained and coarse motor skills simultaneously. Inspired by human motor control, we propose Multi-Resolution Skill Discovery (MRSD), an HRL framework that learns multiple skill encoders at different temporal resolutions in parallel. A high-level manager dynamically selects among these skills, enabling adaptive control strategies over time. We evaluate MRSD on tasks from the DeepMind Control Suite and show that it outperforms prior state-of-the-art skill discovery and HRL methods, achieving faster convergence and higher final performance. Our findings highlight the benefits of integrating multi-resolution skills in HRL, paving the way for more versatile and efficient agents.