When Do Multi-Agent Systems Help? An Information Bottleneck Perspective

This paper frames the comparison between multi-agent systems (MAS) and single-agent systems (SAS) as an information bottleneck problem. A SAS retains its full reasoning trace in one shared context, whereas a MAS connects isolated local contexts through bounded relay messages. The authors argue that unlimited relay bandwidth lets MAS simulate SAS, so any meaningful MAS advantage must come from bounded communication. Across 18 controlled experiments on five benchmarks and three model scales, MAS helps most when relays are nearly sufficient, particularly for weaker models. Gains shrink or reverse when relay messages lose task-relevant information, especially for stronger models that can already exploit redundant context.