CARE-LoRA: Compressed Activation Reconstruction for Memory-Efficient LoRA

CARE-LoRA targets the activation-memory bottleneck that remains after LoRA has reduced trainable parameter count. The method stores the low-rank compressed activation produced by the LoRA branch instead of the full input activation, then computes a lightweight reconstruction matrix during the forward pass to recover the gradient signal during backpropagation. The authors report substantial reductions in overall memory use and competitive or better performance across diverse models and downstream tasks. Source code is publicly available, although the supplied abstract does not provide exact memory or benchmark figures.