LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget

LongStraw is an architecture-aware execution stack for million-token RL post-training under a fixed GPU budget, instantiated with GRPO. It evaluates shared prompts without autograd, retains only model-specific state, and replays short response branches sequentially to reduce the live training graph. On eight H20 GPUs, it handles 2.1M positions for grouped Qwen3.6-27B scoring and response backward, with only 0.21 GB more peak memory when group size grows from 2 to 8. A 32-GPU run validates the execution path for a 2.1M-token GLM-5.2 prompt, but not complete training correctness.