Embarrassingly Simple Self-Distillation Improves Code Generation

A new self-distillation technique significantly boosts LLM code generation performance. Simple Self-Distillation (SSD) fine-tunes models on their own outputs, achieving over 12 percentage point gains on LiveCodeBench for Qwen3-30B-Instruct. This method proves effective across various model sizes and architectures, particularly enhancing performance on challenging coding tasks.

Opening Kapyn…