Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models

This paper introduces function-aware fill-in-the-middle (FIM) mid-training for coding-agent foundation models. It selects masked functions using program dependency graphs and a complexity-inferability criterion, treating function calls as a structural analogue of an agent’s action-observation-continuation loop. The authors mid-train Qwen2.5-Coder-Instruct 7B/14B and Qwen3-8B on a decontaminated 2.6B-token corpus from 968 GitHub repositories. After existing agentic post-training, SWE-Bench-Verified improves by 2.8–3.2 points and SWE-Bench-Lite by 3.7–5.4 points. The reported gains persist across multiple pipelines and appear to reduce capability erosion on non-agent coding and tool-use benchmarks.