MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training

The paper introduces Multi-teacher On-Policy Distillation (MOPD) for integrating capabilities from multiple domain-specific RL teachers. Each teacher is first trained independently through specialised reinforcement learning. The student then generates its own rollouts and distils supervision from the teachers on those on-policy trajectories, providing dense signals while reducing exposure bias. On Qwen3-30B-A3B, the authors report that MOPD outperforms Mix-RL, Cascade RL, Off-Policy Finetuning, and parameter merging, while retaining nearly all teachers’ capabilities. The paper also states that MOPD was used in post-training MiMo-V2-Flash.