HumanCLAW: Can Vision-Language Models Act Through a Body?

HumanCLAW introduces an embodied-AI evaluation framework that separates a vision-language model’s action choices from low-level motor execution. At each step, a harnessed off-the-shelf VLM issues an atomic skill command, which is converted into a sub-second chunk of continuous full-body motion with physical consequences such as gravity and collisions. HumanCLAW-Bench contains 1,218 long-horizon egocentric find-navigate-interact episodes across 41 indoor scenes. Across nine state-of-the-art VLMs, the best success rate is 16.8%, and no model solves the benchmark. The abstract identifies embodied self-awareness, rather than target recognition, as the central weakness.