CCCL: In-GPU Compression-Coupled Collective Communication
For LLM workloads with tensor/expert parallelism, CCCL provides a drop-in replacement for NCCL that improves communication efficiency without code changes.
CCCL introduces a compression-coupled collective communication library that fuses compression with NCCL to reduce memory accesses and eliminate data coalescing, achieving up to 3x NVLink bandwidth and improving vLLM PD disaggregation throughput by up to 10.1%.
Collective communication incurs significant overhead in LLM workloads. Although overlapping communication with computation in application-level is a common strategy, it often requires substantial code modifications and is impractical for many workloads (e.g., tensor and expert parallelism). We present CCCL, a built-in compression-based collective communication library that supports operations such as allreduce, alltoall, and send/recv without requiring any user-side changes, thereby enabling seamless adoption in existing applications. CCCL tightly fuses compression kernels to minimize memory accesses and integrates with NCCL to eliminate the data coalescing stage, making it fast enough (up to 3x NVLink bandwidth) to sustain communication. Our evaluation shows that CCCL improves end-to-end throughput in vLLM PD disaggregation workloads by up to 10.1% and microbenchmark throughput by up to 30%.