Reversing Arrows in Large Language Models

This paper presents a systematic study of whether large language models capture the direction-dependent semantics of inverse relations, such as “mother” versus “child.” The authors introduce a benchmark with 5,457 instances covering 27 inverse-relation labels, evaluate five open-source LLMs with multiple-choice prompts, and test the effects of relation descriptions and entity representations by replacing original entities with synthetic or masked ones. The reported results show systematic asymmetries in inverse-relation classification, inconsistent benefits from relation descriptions, and sensitivity to entity representation changes.