JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition

Most GUI benchmarks evaluate agents inside isolated, static environments, overlooking how workflows actually cross devices. JarvisGUI establishes a unified evaluation framework across Android, Windows, and Ubuntu, formalizing tasks as typed input-output transformations to compose multi-device workflows dynamically. Experiments in multi-OS virtual environments reveal that leading open-source agents struggle with state-transfer awareness and cross-platform contextual reasoning, exposing capability gaps that single-device evaluations hide.