Quantifying Similarity between Relations with Fact Distribution

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2882–2894 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2882 Quantifying Similarity between Relations with Fact Distribution Weize Chen Hao Zhu Xu Han Zhiyuan Liu Maosong Sun Department of Computer Science and Technology, Tsinghua University, Beijing, China State Key Lab on Intelligent Technology and Systems Institute for Artificial Intelligence {wei10,zhuhao15,hanxu17}@mails.tsinghua.edu.cn {liuzy,sms}@tsinghua.edu.cn Abstract We introduce a conceptually simple and effec- tive method to quantify the similarity between relations in knowledge bases. Specifically, our approach is based on the divergence between the conditional probability distributions over entity pairs. In this paper, these distributions are parameterized by a very simple neural network. Although computing the exact similarity is in- tractable, we provide a sampling-based method to get a good approximation. We empirically show the outputs of our ap- proach significantly correlate with human judg- ments. By applying our method to various tasks, we also find that (1) our approach could ef- fectively detect redundant relations extracted by open information extraction (Open IE) models, that (2) even the most competitive models for relational classification still make mistakes among very similar relations, and that (3) our approach could be incorporated into negative sampling and softmax classifi- cation to alleviate these mistakes. The source code and experiment details of this paper can be obtained from https://github.com/ thunlp/relation-similarity. 1 Introduction Relations1, representing various types of connec- tions between entities or arguments, are the core of expressing relational facts in most general knowl- edge bases (KBs) (Suchanek et al., 2007; Bollacker et al., 2008). Hence, identifying relations is a crucial problem for several information extraction tasks. Although considerable effort has been devoted to these tasks, some nuances between similar relations Author contributions: Hao Zhu initiated the research; Weize Chen prepared the data, and organized data annotation; Hao Zhu and Xu Han designed the experiments; Weize Chen performed the experiments; Hao Zhu, Weize Chen and Xu Han wrote the paper; Zhiyuan Liu and Maosong Sun proofread the paper. Zhiyuan Liu is the corresponding author. 1Sometimes relations are also named properties. Sentence The crisis didn’t influence his two daughters OBJ and SUBJ. Correct per:siblings Predicted per:parents Similarity Rank 2 Table 1: An illustration of the errors made by relation ex- traction models. The sentence contains obvious patterns indicating the two persons are siblings, but the model predicts it as parents. We introduce an approach to mea- sure the similarity between relations. Our result shows “siblings” is the second most similar one to “parents”. By applying this approach, we could analyze the errors made by models, and help reduce errors. are still overlooked, (Table 1 shows an example); on the other hand, some distinct surface forms car- rying the same relational semantics are mistaken as different relations. These severe problems motivate us to quantify the similarity between relations in a more effective and robust method. In this paper, we introduce an adaptive and gen- eral framework for measuring similarity of the pairs of …

Content truncated to 500 words