Weakly Supervised Cross-lingual Semantic Relation Classification via Knowledge Distillation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5285–5296, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5285 Weakly Supervised Cross-lingual Semantic Relation Classification via Knowledge Distillation Yogarshi Vyas Department of Computer Science University of Maryland yogarshi@cs.umd.edu Marine Carpuat Department of Computer Science University of Maryland marine@cs.umd.edu Abstract Words in different languages rarely cover the exact same semantic space. This work charac- terizes differences in meaning between words across languages using semantic relations that have been used to relate the meaning of En- glish words. However, because of transla- tion ambiguity, semantic relations are not al- ways preserved by translation. We intro- duce a cross-lingual relation classifier trained only with English examples and a bilingual dictionary. Our classifier relies on a novel attention-based distillation approach to ac- count for translation ambiguity when trans- ferring knowledge from English to cross- lingual settings. On new English-Chinese and English-Hindi test sets, the resulting models largely outperform baselines that more na¨ıvely rely on bilingual embeddings or dictionaries for cross-lingual transfer, and approach the performance of fully supervised systems on English tasks. 1 Introduction Natural Language Processing (NLP) often uses translation lexicons for projecting models and data from one language to another under the assump- tion that words and their translations in these lexicons are synonyms (Mayhew et al., 2017; Tsvetkov et al., 2014). However, translation lex- icons include semantic relations other than syn- onymy in practice, as can be seen (Table 1) in ex- amples drawn from the MUSE dictionary (Lam- ple et al., 2018). Peirsman and Pad´o (2011) show that distributional translation lexicons contain hy- ponyms and co-hyponyms, and that treating all translations as synonyms hurts cross-lingual pro- jection performance. In the Paraphrase Database (Ganitkevitch et al., 2013), Pavlick et al. (2015) find that the diversity of semantic relations discov- ered in word-aligned parallel corpora yields para- phrases that span the lexical relations defined in Lexicon Entry Semantic Relation writer, ������� writer is more specific than ������� (creator) council, �������� council is more general than �������� (council of ministers) father, ��� father is mutually exclusive to ��� (father’s brother) Table 1: Semantic relations between word pairs in an English-Hindi lexicon (Lample et al., 2018) the natural logic framework of MacCartney and Manning (2009). These non-synonymous transla- tions are not just noise. They can be found even in high-quality parallel corpora, since the strategies used by professional translators to deal with words that do not have a direct equivalent in the target language include replacement by near-synonyms, hypernyms or negated antonyms (Baker, 2011; Venuti, 2012; Chesterman, 2016). In this work, we classify semantic relations be- tween words in different languages. Given a word pair (water, ��), the classification task is to select one of the five entailment classes (Figure 1) de- fined under the natural logic framework of Mac- Cartney and Manning (2009). This cross-lingual task cannot be solved by translation, as transla- tion does not …

Content truncated to 500 words