BiRRE: Learning Bidirectional Residual Relation Embeddings for Supervised Hypernymy Detection

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3630–3640 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 3630 BiRRE: Learning Bidirectional Residual Relation Embeddings for Supervised Hypernymy Detection Chengyu Wang1,2, Xiaofeng He3∗ 1 School of Software Engineering, East China Normal University 2 Alibaba Group 3 School of Computer Science and Technology, East China Normal University chywang2013@gmail.com, hexf@cs.ecnu.edu.cn Abstract The hypernymy detection task has been ad- dressed under various frameworks. Previously, the design of unsupervised hypernymy scores has been extensively studied. In contrast, su- pervised classifiers, especially distributional models, leverage the global contexts of terms to make predictions, but are more likely to suffer from “lexical memorization”. In this work, we revisit supervised distributional mod- els for hypernymy detection. Rather than tak- ing embeddings of two terms as classification inputs, we introduce a representation learning framework named Bidirectional Residual Re- lation Embeddings (BiRRE). In this model, a term pair is represented by a BiRRE vector as features for hypernymy classification, which models the possibility of a term being mapped to another in the embedding space by hyper- nymy relations. A Latent Projection Model with Negative Regularization (LPMNR) is pro- posed to simulate how hypernyms and hy- ponyms are generated by neural language mod- els, and to generate BiRRE vectors based on bidirectional residuals of projections. Experi- ments verify BiRRE outperforms strong base- lines over various evaluation frameworks. 1 Introduction As a type of linguistic resources, hypernymy rela- tions refer to “is-a” relations between terms. Such relations are frequently exploited in a wide range of NLP tasks, including taxonomy induction (Mao et al., 2018), lexical entailment (Vulic et al., 2017) and Web query understanding (Wang et al., 2015). In the NLP community, the task of hyper- nymy detection has been studied under various frameworks, e.g., unsupervised hypernym dis- covery (Roller et al., 2018; Chen et al., 2018; Chang et al., 2018), supervised hypernymy clas- sification (Shwartz et al., 2016; Nguyen et al., ∗ Corresponding author. 2017), graded lexical entailment (Vulic et al., 2017). To address unsupervised hypernym discov- ery, pattern-based and distributional approaches are two mainstream types of methods. Pattern-based approaches use Hearst patterns (Hearst, 1992) and their variants to extract hypernymy relations from texts (Kozareva and Hovy, 2010; Roller and Erk, 2016). Distributional methods employ hypernymy measures (or called scores) to predict hypernymy based on distributional vectors (Santus et al., 2014, 2017), alleviating the pattern sparsity issue. Le et al. (2019) combine Hearst patterns and hyperbolic em- beddings for unsupervised hypernym detection. Compared to unsupervised tasks, the supervised hypernymy detection task is formulated more di- rectly, classifying a term pair as hypernymy or non-hypernymy based on two terms’ representa- tions (Yu et al., 2015; Anke et al., 2016; Nguyen et al., 2017). Although this task definition is more straightforward, the corresponding methods receive criticism because they may suffer from “lexical memorization” (Levy et al., 2015), referring to the phenomenon that they only learn whether a term is a “prototypical hypernym”, rather than the ac- …

Content truncated to 500 words