How Does mHC Use Its Residual Streams? Selective Routing and Near-Identity Mixing

Manifold-constrained Hyper-Connections (mHC) expand a transformer’s residual highway into multiple concurrent streams, yet how models exploit this capacity has remained opaque. Analyzing DeepSeek-V4-Flash's four-stream setup reveals that individual attention and FFN blocks typically route through only two streams at a time, while inter-stream mixing occurs primarily in early stages. Across layers 22 through 42, the pathway behaves close to an identity operation; replacing late-stage mixers with pure identity increases C4 perplexity by merely 1.9%, indicating that multi-stream residual capacity is substantially underused in deeper layers.