Entangled by Design: Spurious Intra-Variable Signal Routing in Tabular In-Context Learners

This paper studies spurious routing in tabular in-context learners when one composite feature contains both a causal signal C and an environment-specific artefact S. In a single training environment, the learner may route predictions through S because the in-context data cannot identify which subspace is causal. The authors prove this behaviour is unavoidable under ridge ICL, report qualitatively consistent results with TabPFN, and propose environment-stratified contexts and S-swap augmentation. S-swap reduces spurious routing by 74% for linear ICL and 98.8% for TabPFN while increasing causal sensitivity 8.4-fold.