Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text

The paper introduces ProVisE, a benchmark-agnostic framework for evaluating image-generation models through protocol-constrained visual answers such as pointing, marking, and drawing. These pixel-space responses are parsed into structured predictions compatible with existing benchmark metrics. It also presents SpatialGen-Bench, containing 470 samples across 14 spatial subtasks, four capability levels, and diverse answer formats. According to the abstract, image-generation models are competitive when spatial judgments can be externalized directly in pixels, while text-output VLMs maintain a clear advantage in compositional spatial reasoning. An agentic builder is additionally used to construct and validate task-specific protocols across six external spatial benchmarks.