90/30 Club (ML reading) # Week 43: On-Policy Self-Distillation for Reasoning Compression

​Week 43: On-Policy Self-Distillation for Reasoning Compression

​ The Paper Link here: https://arxiv.org/abs/2603.05433 Read the paper and you will be up to speed! On-Policy Self-Distillation for Reasoning Compression addresses the critical bottleneck in modern reasoning models: the tendency to "think out loud" with excessive, noisy, and often harmful token generation. While previous compression methods relied on ground-truth answers, strict token budgets, or complex difficulty estimators, this work introduces a purely self-distilled approach (OPSDC). It reduces the process to a single idea: conditioning the model on a "be concise" instruction to obtain teacher logits, and minimizing per-token reverse KL on the student's own rollouts. The study demonstrates that OPSDC is highly effective—automatically compressing easy problems aggressively while preserving the deep deliberation needed for hard ones. The authors show that on Qwen3-8B and Qwen3-14B, the method achieves a massive 57–59% token reduction on MATH-500 while simultaneously improving accuracy by 9–16 absolute points. The findings reveal a stark contrast to typical reasoning assumptions: much of what reasoning models produce is not just redundant, but actively harmful, compounding errors with every unnecessary token. Join us at Mox to explore: - How the OPSDC method leverages a simple "be concise" instruction and reverse KL to distill reasoning without ground-truth answers. Why on-policy self-distillation automatically calibrates token usage, compressing easy problems while maintaining deliberation for hard ones. The "Harmful Redundancy" problem: why extra tokens in reasoning models are often not just noise, but actively compound errors. 🔎Analyzed Papers Discussion at 20:00, (optional) quiet reading from 19:00. On-Policy Self-Distillation for Reasoning Compression, link here --> https://arxiv.org/abs/2603.05433