SemPIC: Learning Semantic Position-Independent KV Caches

SemPIC targets repeated document reuse in long-context retrieval and agentic workloads where instructions, histories, and document order change. A LoRA-enabled Writer compiles native per-layer document KV caches offline through behavioral distillation, while the pretrained decoder remains an unchanged Reader. The standard KV interface and cache-hit decoding path are preserved. The paper also introduces KV Gradient Checkpointing to reduce peak training memory while maintaining gradients through cached KVs. Across three models and four tasks, mean micro-F1 increases from 0.53 for KV Packet to 0.60, approaching 0.62 for Full Recompute.