DRIFT: Derailing Denoising Trajectories of Flow-Matching VLAs with Adversarial Patch Attack

The paper introduces DRIFT, a test-time universal adversarial patch placed on a robot gripper to redirect the denoising trajectory of flow-matching vision-language-action models. Rather than optimizing attacks across many denoising steps, DRIFT targets only the first step. The authors report that this is both cheaper and more effective, attributing the result to gradient conflict in input-space optimization. On pi0 and pi0.5 across four LIBERO suites, the attack reportedly breaks essentially all tasks that were originally solvable and substantially outperforms action-space and embedding-space baselines.