Topology-Aware Revival for Efficient Sparse Training
This work addresses the brittleness of static sparse training in deep reinforcement learning, offering an incremental improvement for efficient learning in continuous-control tasks.
The paper tackles the problem of static sparse training's reduced robustness in deep reinforcement learning by proposing Topology-Aware Revival (TAR), a one-shot post-pruning procedure that improves final return by up to +37.9% over static sparse baselines and achieves a median gain of +13.5% over dynamic sparse training baselines.
Static sparse training is a promising route to efficient learning by committing to a fixed mask pattern, yet the constrained structure reduces robustness. Early pruning decisions can lock the network into a brittle structure that is difficult to escape, especially in deep reinforcement learning (RL) where the evolving policy continually shifts the training distribution. We propose Topology-Aware Revival (TAR), a lightweight one-shot post-pruning procedure that improves static sparsity without dynamic rewiring. After static pruning, TAR performs a single revival step by allocating a small reserve budget across layers according to topology needs, randomly uniformly reactivating a few previously pruned connections within each layer, and then keeping the resulting connectivity fixed for the remainder of training. Across multiple continuous-control tasks with SAC and TD3, TAR improves final return over static sparse baselines by up to +37.9% and also outperforms dynamic sparse training baselines with a median gain of +13.5%.