This paper reframes norm-preserving residual updates on a hypersphere as a scalar choice that converts update magnitude into a rotation angle. It introduces Proj-SpheretNorm and Cay-SpheretNorm, algebraic alternatives to the exponential-map-based GeoNorm, and places them in a one-parameter angular-retraction family called p-SpheretNorm. The supplied abstract states that p=1 and p=2 recover the projection and Cayley variants, while GeoNorm and the identity appear only as limiting cases. On nanoGPT, all three proposed methods reportedly outperform existing lightweight deep-connection schemes, with the best validation loss occurring at finite p.
No heat snapshots are available in the last 24 hours.