CAPEval: Decoupled Caption Evaluation for Understanding and Generation

CAPEval proposes evaluating image captions through two separate dimensions: Coverage, which measures how thoroughly a caption captures factual visual content, and Precision, which measures the fraction of its claims that are supported by the image. The benchmark uses human-written ground-truth captions and human-verified atomic checklist items. Experiments compare 10 captioners from four model families under controlled downstream settings where caption source is the only changing variable. The reported results show that Coverage is more strongly associated with multimodal understanding, while Precision is the dominant predictor of text-to-image generation performance.