Unlocking Software-defined GPU Fabric Scheduling in the LLM Era

Abstract

Large language model (LLM) systems increasingly rely on aggressive optimizations such as prefill-decode (PD) disaggregation, KV cache offloading, and compute–communication overlap to maximize hardware utilization and meet strict latency and throughput requirements. However, these techniques implicitly assume that the underlying GPU interconnects behave as neutral communication substrates. In practice, modern GPU clusters contain shared arbitration domains across PCIe, NVLink, and RDMA fabrics, where concurrent data movement contends for the same resources. In this paper, we characterize fabric contention in modern GPU clusters and find that it can expose implicit hardware priorities among traffic classes, leading to performance behavior that deviates from application demands. In particular, GPU-to-GPU communication over NVLink can contend with local GPU memory access and slow compute progress during communication–computation overlap, while RDMA communication can dominate PCIe arbitration and substantially degrade host–device data movement. To address this challenge, we propose GPUWeaver, a lightweight runtime framework for microarchitecture-aware communication scheduling. GPUWeaver monitors application progress and fabric utilization, detects emerging contention, and dynamically regulates communication injection to better align resource usage with application demands. In a vLLM+Mooncake case study using Qwen3-8B and request shapes from a real trace, GPUWeaver reduces the PD KV transfer flow time by 19.4-34.4% and mean TTFT by 2.4-6.1%.