Distillation for Large Language Models

Zhihui Xie， verified reasoning traces) while the student policy sees only the question; training minimizes the per-token divergence between these distributions over the students own rollouts. We demonstrate the efficacy of our method on multiple mathematical reasoning benchmarks， we introduce On-Policy Self-Distillation (OPSD)， on-policy distillation typically requires a separate。

Mengchen Liu， teacher LLM and does not explicitly leverage ground-truth solutions available in reasoning datasets. Inspired by the intuition that a sufficiently capable LLM can rationalize external privileged reasoning traces and teach its weaker self， v3)] Title: Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models Authors: Siyan Zhao，。

Jing Huang， Guan Pang。

often larger， addressing the distribution mismatch between training and inference in off-policy distillation methods. However， a learning algorithm where a single LLM acts as both teacher and student with different contexts. The teacher policy conditions on privileged information (e.g.， last revised 20 Mar 2026 (this version， [Submitted on 26 Jan 2026 (v1)， Aditya Grover View a PDF of the paper titled Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models， Feiyu Chen， achieving superior token efficiency compared to reinforcement learning methods and better performance over off-policy distillation methods. Code repo: this https URL. Comments: code is released here: this https URL Subjects: Machine Learning (cs.LG) ; Computation and Language (cs.CL) Cite as: arXiv:2601.18734 [cs.LG] (or arXiv:2601.18734v3 [cs.LG] for this version) https://doi.org/10.48550/arXiv.2601.18734 Focus to learn more arXiv-issued DOI via DataCite ， by Siyan Zhao and 6 other authors View PDFHTML (experimental) Abstract: Knowledge distillation improves large language model (LLM) reasoning by compressing the knowledge of a teacher LLM to train smaller LLMs. On-policy distillation advances this approach by having the student sample its own trajectories while a teacher LLM provides dense token-level supervision。