Improve TTFT for multi-turn agentic workloads with bi-directional KV transfer in vLLM

vLLM introduces bi‑directional KV cache transfer to speed up multi‑turn agentic workloads. The technique lets prefill and decode nodes reuse context, cutting time to first token by up to 1.6× for Qwen3‑32B and 3.3× for GLM‑5‑FP8 on Amazon EC2 P5en with EFA. This boosts latency‑sensitive LLM applications.

Opening Kapyn…