Learning When to Stop: Prefix-Optimal Dynamic Diffusion Policies for Continuous Control
The paper introduces Prefix-Optimal Generative Policies (POGP), which learn a prefix value function at every intermediate denoising step of a diffusion policy. The function provides an auxiliary training signal for producing useful intermediate actions and supports test-time early stopping when further denoising is unlikely to help. Across four MuJoCo environments and 12 baselines, POGP reduces required denoising iterations by approximately 2.7-fold while retaining near-full task performance. Against state-of-the-art dynamic diffusion baselines, prefix training improves final task performance by approximately 3.5%.
Why it's worth reading
Diffusion-policy inference cost is a practical bottleneck for continuous control. This work reports roughly 2.7-fold fewer denoising iterations while preserving performance, making its stopping criterion and experimental boundaries timely to examine.