Research note · July 20, 2026

On-Policy Distillation: Discussion of Loss Design

A practical look at reverse KL, better estimators, Top-K truncation, and why representation distillation still struggles in on-policy settings.