Mind the Labels: Describing Relations in Knowledge Graphs With Pretrained Models

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2398–2415 May 2-6, 2023 ©2023 Association for Computational Linguistics Mind the Labels: Describing Relations in Knowledge Graphs With Pretrained Models Zdenˇek Kasner,1 Ioannis Konstas2 and Ondˇrej Dušek1 1Charles University, Faculty of Mathematics and Physics, Prague, Czechia 2The Interaction Lab, MACS, Heriot-Watt University, Edinburgh, UK {kasner,odusek}@ufal.mff.cuni.cz, i.konstas@hw.ac.uk Abstract Pretrained language models (PLMs) for data-to- text (D2T) generation can use human-readable data labels such as column headings, keys, or relation names to generalize to out-of-domain examples. However, the models are well- known in producing semantically inaccurate outputs if these labels are ambiguous or incom- plete, which is often the case in D2T datasets. In this paper, we expose this issue on the task of descibing a relation between two entities. For our experiments, we collect a novel dataset for verbalizing a diverse set of 1,522 unique rela- tions from three large-scale knowledge graphs (Wikidata, DBPedia, YAGO). We find that al- though PLMs for D2T generation expectedly fail on unclear cases, models trained with a large variety of relation labels are surprisingly robust in verbalizing novel, unseen relations. We argue that using data with a diverse set of clear and meaningful labels is key to training D2T generation systems capable of generaliz- ing to novel domains.1 1 Introduction D2T generation systems need to accurately cap- ture the semantics of relations between values in the data. However, the data labels such as rela- tion names (Färber et al., 2018; Haller et al., 2022), table headings (Parikh et al., 2020), or meaning rep- resentation keys (Dušek et al., 2020) may provide only superficial or—if the labels are abbreviations, such as in the Rotowire dataset (Wiseman et al., 2017)—no usable hints about the data semantics. Learning how to properly describe the data is thus a challenge for D2T systems, typically requiring in-domain training data of sufficient quality and quantity (Dušek et al., 2019). PLMs such as BART (Lewis et al., 2020) or T5 (Raffel et al., 2020) can quickly adapt to new domains and exhibit robustness to out-of-domain 1We release the code and data for our experiments: https://github.com/kasnerz/rel2text. model ref lex sem Werner Körte is the godparent of Rudolf Virchow. model ref Rudolf Virchow is the godparent of Werner Körte. Deepdale's occupant is Chorley Lynx. model ref Chorley Lynx plays at Deepdale. lex sem lex sem (c) (b) (a) Dan Jones composed the music for My Scientology Movie. Dan Jones composed the sound- track for My Scientology Movie. godparent Werner Körte Rudolf Virchow occupant Deepdale Chorley Lynx musicBy Dan Jones My Scientology Movie Figure 1: Data-to-text generation models use relation labels (such as godparent, occupant, and musicBy) to describe relations between entities. However, unclear labels can lead to various lexical or semantic incoheren- cies in the output descriptions, such as swapping the relation direction (a) or using too literal expressions (b). inputs. However, the PLMs for D2T generation are still limited by the expressivity of the data labels. Consider Figure …

Content truncated to 500 words