Understanding Reasoning from Pretraining to Post-Training

This paper studies the interface between pretraining and reinforcement-learning post-training using chess as a controlled testbed. Models ranging from 5M to 1B parameters are pretrained on human chess games, supervised-finetuned on synthetic reasoning traces, and trained with verifiable-reward RL on chess puzzles. The authors report that pretraining loss predicts post-RL performance at a given RL compute budget, while the slope of RL reward curves improves approximately linearly with pretraining tokens. RL reinforces already-preferred correct moves on easy puzzles but can uncover correct moves nearly absent from the SFT policy on hard puzzles. A 1B math-domain model shows a similar pattern.