The paper introduces Hyper-Spherical Quantization (HSQ) for discretizing high-dimensional visual representations in a Representation Autoencoder. It argues that Euclidean codebook objectives mismatch the anisotropic geometry of representation spaces, causing magnitude-dominated assignments, codebook collapse, and uneven angular coverage. HSQ routes codes by angular information while decoupling semantic content from feature magnitude. The resulting dRAE reportedly maintains high-fidelity reconstruction, reaches 100% codebook utilization, and scales to a vocabulary of 131,072 codes, with gains across understanding and generation tasks and a simplified training pipeline.
No heat snapshots are available in the last 24 hours.