MANCE: Manifold Aware Concept Erasure

The paper proposes the Manifold Constraint Hypothesis: because natural representations may concentrate on structured, lower-dimensional manifolds, concept-erasure updates should remain on those manifolds to preserve unrelated information. MANCE estimates a manifold from representations of natural inputs, then projects classifier-guided erasure updates onto it. Across 119 text and vision settings involving 13 language models, three NLP concepts, and 40 CelebA-CLIP attributes, adding MANCE to prior methods consistently improves leakage results. MANCE+ and MANCE++ prepend closed-form erasure, with MANCE++ reporting state-of-the-art results for nonlinear concept erasure.