Cross-lingual Semantic Specialization via Lexical Relation Induction

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2206–2217, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2206 Cross-lingual Semantic Specialization via Lexical Relation Induction Edoardo M. Ponti1, Ivan Vuli´c1, Goran Glavaˇs2, Roi Reichart3, Anna Korhonen1 1Language Technology Lab, TAL, University of Cambridge 2Data and Web Science Group, University of Mannheim, Germany 3Faculty of Industrial Engineering and Management, Technion, IIT 1{ep490,iv250,alk23}@cam.ac.uk 2goran@informatik.uni-mannheim.de 3roiri@ie.technion.ac.il Abstract Semantic specialization integrates structured linguistic knowledge from external resources (such as lexical relations in WordNet) into pretrained distributional vectors in the form of constraints. However, this technique can- not be leveraged in many languages, because their structured external resources are typically incomplete or non-existent. To bridge this gap, we propose a novel method that trans- fers specialization from a resource-rich source language (English) to virtually any target lan- guage. Our specialization transfer comprises two crucial steps: 1) Inducing noisy con- straints in the target language through auto- matic word translation; and 2) Filtering the noisy constraints via a state-of-the-art relation prediction model trained on the source lan- guage constraints. This allows us to special- ize any set of distributional vectors in the tar- get language with the refined constraints. We prove the effectiveness of our method through intrinsic word similarity evaluation in 8 lan- guages, and with 3 downstream tasks in 5 languages: lexical simplification, dialog state tracking, and semantic textual similarity. The gains over the previous state-of-art specializa- tion methods are substantial and consistent across languages. Our results also suggest that the transfer method is effective even for lexically distant source-target language pairs. Finally, as a by-product, our method pro- duces lists of WordNet-style lexical relations in resource-poor languages. 1 Introduction Due to their dependence on the distributional hy- pothesis (Harris, 1954), that is, word co-occurrence information in large corpora, distributional word embeddings (Mikolov et al., 2013; Levy and Gold- berg, 2014; Pennington et al., 2014; Melamud et al., 2016; Bojanowski et al., 2017; Peters et al., 2018, inter alia) conflate paradigmatic relations (e.g., synonymy, antonymy, lexical entailment, co- hyponymy, meronymy) and the broader topical (i.e., syntagmatic) relatedness (Schwartz et al., 2015; Mrkˇsi´c et al., 2017). This property can propagate undesired effects to language understanding appli- cations such as statistical dialog modeling or text simplification (Faruqui, 2016; Chiu et al., 2016; Mrkˇsi´c et al., 2017): for instance, the inability to distinguish between synonymy and antonymy (e.g., between cheap pubs and expensive restaurants) can break task-oriented dialog or a recommendation system (Mrkˇsi´c et al., 2016; Kim et al., 2016b). Semantic specialization techniques are therefore leveraged to stress a relation of interest such as semantic similarity (Wieting et al., 2015; Mrkˇsi´c et al., 2017; Ponti et al., 2018) or lexical entailment (Nguyen et al., 2017; Vuli´c and Mrkˇsi´c, 2018) over other types of semantic association in the word vector space. The best-performing specialization models (cf. Mrkˇsi´c et al. 2017; Ponti et al. 2018) are executed as vector space post-processors. …

Content truncated to 500 words