ARISE: Adaptive Reinforcement Integrated with Swarm Exploration
This work addresses the problem of exploration for reinforcement learning practitioners, offering a simple, architecture-agnostic solution that is incremental as it builds upon standard policy-gradient methods.
The paper tackles the challenge of effective exploration in reinforcement learning, especially with non-stationary rewards or high-dimensional policies, by introducing ARISE, a lightweight framework that enhances policy-gradient methods with swarm-based exploration, resulting in substantial gains such as +46% on LunarLander-v3 and +22% on Hopper-v4, while providing marked robustness advantages like outperforming PPO by +75 points on CartPole under non-stationary reward shifts.
Effective exploration remains a key challenge in RL, especially with non-stationary rewards or high-dimensional policies. We introduce ARISE, a lightweight framework that enhances reinforcement learning by augmenting standard policy-gradient methods with a compact swarm-based exploration layer. ARISE blends policy actions with particle-driven proposals, where each particle represents a candidate policy trajectory sampled in the action space, and modulates exploration adaptively using reward-variance cues. While easy benchmarks exhibit only slight improvements (e.g., +0.7% on CartPole-v1), ARISE yields substantial gains on more challenging tasks, including +46% on LunarLander-v3 and +22% on Hopper-v4, while preserving stability on Walker2d and Ant. Under non-stationary reward shifts, ARISE provides marked robustness advantages, outperforming PPO by +75 points on CartPole and improving LunarLander accordingly. Ablation studies confirm that both the swarm component and the adaptive mechanism contribute to the performance. Overall, ARISE offers a simple, architecture-agnostic route to more exploratory and resilient RL agents without altering core algorithmic structures.