Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation

This paper argues that directly matching classifier-free-guidance (CFG) composed velocities in on-policy diffusion distillation is under-identified at the branch level: positive- and negative-condition errors can compensate in the composed prediction. It identifies a failure mode called Negative Branch Asymmetry (NBA), which arises when the teacher’s negative branch contains privileged information unavailable to the student. The authors propose Positive-Direction Matching (PDM), which separately constrains the positive prediction and the CFG conditional direction. In dense-to-sparse video control, PDM reportedly makes knowledge transfer more robust to inference guidance scales than naive guided matching.