ShortOPD addresses a deployment gap in structured-pruned LLMs: checkpoints that remain acceptable on multiple-choice recognition can collapse during free-form generation, with greedy pass@1 nearly disappearing and repetitive suffixes dominating recovery. The method applies on-policy distillation using the compressed model’s own states and a frozen pre-compression teacher, while scheduling rollout lengths from short to long according to teacher-confirmed effective prefixes. The abstract reports results across math, code, and open-ended generation: roughly 9x the unrecovered score and 1.6–4.4x standard recovery recipes. It reaches within two points of a fixed 8192-token horizon using 8.5 rather than 35.9 training hours and 71% fewer rollout tokens.
No heat snapshots are available in the last 24 hours.