On-Policy Distillation: Discussion of Loss Design

A practical look at reverse KL, better estimators, Top-K truncation, and why representation distillation still struggles in on-policy settings.