PAR: Progress-Aligned Recurrence for Length Generalization in State Tracking

While looped language models can execute additional recurrent iterations at inference time, added depth rarely translates cleanly into solving longer sequential tasks. The Progress-Aligned Recurrence (PAR) framework resolves this disconnect by aligning the k-th iteration directly with the k-th state transition while rescaling recurrent hidden states to their initial norm. On sequential state-tracking benchmarks with Qwen2.5-0.5B, models trained on sequences up to 32 moves retained a 95.5% accuracy rate at 512 moves—achieving a 16× extrapolation factor that sharply outpaces standard recurrent baselines and explicit chain-of-thought.