DynaResize: Runtime GPU Reallocation for Disaggregated LLM Post-Training
For practitioners of RL-based LLM post-training, DynaResize addresses pipeline bubbles caused by long-tail rollout latency, offering a runtime solution that improves resource utilization without altering RL semantics.
DynaResize dynamically reallocates GPUs between Rollout and Training stages in disaggregated LLM post-training, improving throughput by 66.5% and reducing execution time by 33% over optimal static configuration.
RL-based LLM post-training increasingly disaggregates Rollout and Training across separate GPU resources, but static GPU partitioning suffers from severe pipeline bubbles under long-tail rollout latency. We present DynaResize, a runtime GPU reallocation system that dynamically switches GPUs between Rollout and Training to balance stage execution times without changing RL semantics. DynaResize decomposes resizing into fine-grained operations and removes non-startup-critical work from the critical path through communicator reuse, bounded state staging, and hysteresis-based resizing. Experimental results show that DynaResize can improve end-to-end throughput by 66.5% and reduce total execution time by 33% over the optimal static configuration, while hiding 27% of role-switching overhead.