Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM

vLLM introduces a push‑based NIXL connector that offloads KV cache transfer from the critical path, cutting TTFT by up to 3× and improving TPOT by 30% on Amazon EC2 P5en with AWS EFA. This boosts disaggregated inference performance, making large‑model deployment faster and more efficient for developers.

Opening Kapyn…