Do Maps Still Matter for Machines: Revisiting the Role of Choropleth Maps in Foundation Model Spatial Understanding

This paper introduces ChoroplethMap-Bench, a controlled benchmark with 2,400 synthetic choropleth maps, matching GeoJSON data, and 12,000 questions spanning five dimensions: Identify, Spatial Recognition, Compare, Rank, and Delineate. It evaluates 22 open-source and proprietary models under Data Only, Map Only, and Data + Map conditions. According to the supplied abstract, Data + Map performs best overall, with maps especially improving higher-level spatial pattern reasoning when paired with symbolic data. The study also examines map type, hue, spatial structure, prompting, language, geographic context, decoding, classification, and response stability.