xHC: Expanded Hyper-Connections for Practical Large-N Residual Streams

The paper introduces Expanded Hyper-Connections (xHC), targeting two bottlenecks that limit Manifold-Constrained Hyper-Connections (mHC) beyond four residual streams: insufficient write-back information and cubic-cost residual mixing. xHC uses temporal feature augmentation and a sparse residual-stream design that updates only k=4 of N=16 streams while retaining dense access to the full state. On an 18B MoE model, it reports a 4.0-point average downstream improvement over mHC. The accompanying xHC-Flash implementation reduces per-sublayer memory traffic from 73.5C to 40C, approaching mHC’s 34C at N=4.