Mixture-of-Experts kernel engineering on Amazon SageMaker AI

AWS engineers optimize Mixture‑of‑Experts kernels on SageMaker AI, boosting throughput by up to 10% and reducing first‑token latency by 9% on H200 GPUs. The changes include Swap‑AB for WGMMA, Triton backend routing for FP8 quantization, and removal of Triton JIT recompilation, improving performance for large‑scale MoE workloads.

Opening Kapyn…