When Teachers Mislead: Spurious-Signal-Aware On-Policy Distillation

This paper identifies a failure mode in on-policy distillation (OPD): dense token-level teacher signals may be driven by language priors, formatting conventions, or stereotyped reasoning templates rather than task-specific evidence. Such signals can create large but unhelpful optimization updates. The proposed SA-OPD framework estimates token-level input-groundedness and filters only tokens that combine low input dependence with extreme distillation divergence. According to the abstract, experiments across both large language model (LLM) and vision-language model (VLM) settings show consistent gains over Vanilla OPD and competing selective-distillation methods.