PCSD: Persistent Consistency for Self-Distillation in Agentic Reinforcement Learning

PCSD addresses sparse-reward training for language-model agents by deriving token-level self-distillation weights from the local persistence of signals favoring a privileged teacher. It combines adaptive windows, exponentially decayed aggregation, trend-aware attenuation, and sigmoid gating, then jointly optimizes the resulting objective with GRPO. According to the supplied abstract, PCSD leads ALFWorld Overall on two backbones, improving over GRPO by 15.6 and 13.3 points and over SDAR by 6.2 and 5.5 points, while adding 15.8 points over GRPO on an unseen ALFWorld split.