KGCQual: An Interpretable Framework for Evaluating Knowledge Graph Construction Quality from Text

KGCQual is a model-agnostic, interpretable metric for intrinsic evaluation of knowledge graphs automatically extracted from text. It assesses entity completeness, resolution quality, connectivity, predicate preservation, and relation multiplicity, while using lexical similarity, dependency-parse alignment, and lightweight negation handling to approximate an ideal graph grounded in the source text. The authors evaluate it across extraction systems and datasets including WebNLG, TinyButMighty, and BenchIE. They report that KGCQual detects omissions, redundancy, and structural deviations missed by existing metrics, and that its scores significantly correlate with link prediction performance on the extracted graphs.