Semigroup-JEPA: Latent Dynamics Consistency for Zero-Shot Physics Generalization
Evaluating world models under qualitative physical shifts, SG-JEPA extends the JEPA framework to achieve zero-shot generalization across varying gravitational regimes. By conditioning on physics-governing parameters and back-propagating autoregressive rollout loss into the encoder, the method cuts 2D open-loop prediction error by half and improves 3D robotic manipulation success by up to 2.5 times over DINO-WM. Theoretical analysis indicates that performance gains stem primarily from the encoder retaining rollout-consistent representations rather than the predictor merely fitting dynamic trajectories.
Why it's worth reading
It addresses physical out-of-distribution generalization in JEPA world models, showing analytically and empirically that back-propagating rollout loss forces the representation encoder to retain dynamically consistent features.