SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them
AI Summary
SpatialCLI trains vision-language models to use specialist spatial tools and progressively internalize their capabilities. Its three stages are Call, Learn, and Internalize: models first invoke tools for localization, segmentation, depth, and pose, then improve tool use through cold-start SFT and agentic RL, and finally verbalize successful trajectories. On the 516-example SpatialCLI-Bench, the framework targets compositional perception. On MindCube, Qwen3-VL-8B-Instruct improves from 29.3% to 84.6% with tools, exceeding the reported 72.1% for GPT-5.6 Sol with tools, while retaining 73.8% without tools after internalization.
Why it's worth reading
The paper measures both immediate tool augmentation and post-training internalization, making it timely for evaluating whether visual agents can move beyond tool calling toward durable spatial competence.
Deep Read
1. What happened
Original facts: The paper introduces SpatialCLI, a three-stage framework for compositional spatial perception. Call exposes specialist vision models as tools; Learn improves tool use with cold-start SFT and agentic RL; Internalize verbalizes successful tool-use trajectories so the VLM can absorb specialist capabilities.
Analysis: The framework targets a capability mismatch: general VLMs can make task-level decisions but miss decisive visual details, while specialist models perceive details without translating them into actions.
2. Core technology
Original facts: SpatialCLI uses tools for localization, segmentation, depth, and pose. Training first supplements perception externally, then teaches the model when and how to invoke tools, and finally trains it to reproduce useful perceptual reasoning without tool access.
Analysis: This combines tool-augmented inference with trajectory-based distillation. The external tools act both as inference modules and as sources of behavioral supervision.
3. Key evidence and numbers
Original facts: SpatialCLI-Bench contains 516 examples covering compositional perception across localization, segmentation, depth, and pose. On MindCube, Qwen3-VL-8B-Instruct rises from 29.3% to 84.6% with tools. The abstract reports 72.1% for GPT-5.6 Sol with tools. After internalization, Qwen3-VL-8B-Instruct reaches 73.8% without tools.
Analysis: The 29.3% to 84.6% increase indicates that external perception tools address a substantial bottleneck. The 73.8% no-tool result suggests that part of the gain persists after training, although tool access remains better in the reported setting.
4. Why it matters
Analysis: Embodied tasks require both precise perception and task-level decisions. SpatialCLI offers a measurable recipe for decomposing those capabilities: give perception to specialists first, then test whether the main model learns tool-use policies and internal representations.
Unverified inference: If the approach transfers across environments, tools, and longer task horizons, it could reduce dependence on high-latency external vision services. The abstract does not establish this.
5. Practical impact
Analysis: Practitioners could wrap existing localization, segmentation, depth, and pose models as callable tools, use them for high-accuracy execution, and train on successful trajectories to improve autonomous behavior. Evaluation should report both tool-enabled and tool-free performance.
Original facts: The abstract reports benchmark scores for tool use and post-internalization performance, but does not provide latency, invocation counts, or deployment cost.
6. Limitations and uncertainty
Original facts: The supplied information does not specify the benchmark distribution, training-set size, specialist model identities, RL configuration, ablations, variance, or complete baseline protocol.
Analysis: A 516-example benchmark can support focused evaluation but cannot by itself establish robustness in diverse embodied environments. The MindCube gains may depend on tool interfaces, data overlap, prompt templates, or evaluation design.
Unverified inference: It remains unclear whether internalization reflects durable spatial representations rather than memorization of tool trajectories or task templates. Cross-dataset, cross-tool, and out-of-distribution tests are needed.
7. Original sources
- arXiv abstract page
- Source feed: hf-papers
- Model names, method details, and scores in this entry are taken from the user-provided abstract. No authors, institutions, citations, or experimental details beyond that text were added.