Emergent Hierarchical Monosemantic Neurons from the Group-Contrastive Forward-Forward Algorithm

This paper introduces Group-Contrastive Forward-Forward (GCFF), a forward-forward learning algorithm that combines class-specific routing with within-class contrastive objectives. The authors argue that monosemantic neurons can emerge through architectural constraints rather than sparsity. Applied to CLIP representations, one GCFF module reportedly discovers neurons whose abstraction increases with depth, eventually capturing environmental properties independent of image foreground. GCFF can also train networks from scratch and is reported to achieve state-of-the-art results among forward-forward methods on several image-classification benchmarks.