This paper analyzes on-policy distillation, detailing its benefits and drawbacks for training reasoning models. It investigates crucial factors like teacher model selection and context for self-distillation, aiming to clarify optimal strategies for per-token supervision. The research introduces a training-free method to understand these dynamics without expensive experimentation.
Opening Kapyn…