CBA: Communication-Bound-Aware Cross-Domain Resource Assignment for Pipeline-Parallel Distributed LLM Training in Dynamic Multi-DC Optical Networks
This addresses the challenge of efficient resource allocation for large-scale AI training across distributed data centers, which is an incremental improvement over existing methods.
The paper tackles the problem of optimizing pipeline-parallel distributed LLM training in dynamic multi-datacenter optical networks by proposing a communication-bound-aware cross-domain resource assignment framework, resulting in a 31.25% reduction in iteration time and a 13.20% decrease in blocking requests compared to baselines.
We propose a communication-bound-aware cross-domain resource assignment framework for pipeline-parallel distributed training over multi-datacenter optical networks, which lowers iteration time by 31.25% and reduces 13.20% blocking requests compared to baselines.