From Text to Lexicon: Bridging the Gap between Word Embeddings and Lexical Resources

Proceedings of the 27th International Conference on Computational Linguistics, pages 233–244 Santa Fe, New Mexico, USA, August 20-26, 2018. 233 From Text to Lexicon: Bridging the Gap between Word Embeddings and Lexical Resources Ilia Kuznetsov and Iryna Gurevych Ubiquitous Knowledge Processing Lab (UKP-TUDA) Department of Computer Science Technische Universit¨at Darmstadt http://www.ukp.tu-darmstadt.de/ Abstract Distributional word representations (often referred to as word embeddings) are omnipresent in modern NLP. Early work has focused on building representations for word types, and recent studies show that lemmatization and part of speech (POS) disambiguation of targets in isolation improve the performance of word embeddings on a range of downstream tasks. However, the reasons behind these improvements, the qualitative effects of these operations and the combined performance of lemmatized and POS disambiguated targets are less studied. This work aims to close this gap and puts previous findings into a general perspective. We examine the effect of lemmatization and POS typing on word embedding performance in a novel resource-based evaluation scenario, as well as on standard similarity benchmarks. We show that these two operations have complementary qualitative and vocabulary-level effects and are best used in combination. We find that the improvement is more pronounced for verbs and show how lemmatization and POS typing implicitly target some of the verb-specific issues. We claim that the observed improvement is a result of better conceptual alignment between word embeddings and lexical resources, stressing the need for conceptually plausible modeling of word embedding targets. 1 Introduction Word embeddings learned from unlabeled corpora are one of the cornerstones of modern natural language processing. They offer important advantages over their sparse counterparts, allowing efficient computation and capturing a surprising amount of lexical information about words based solely on usage data. Since precise word-related terminology is important for this paper, we introduce it early on. A token is a unique word occurrence within context. Tokens that are represented by the same sequence of characters belong to the same word type. A type signifies one or — in case of morphological ambiguity — several word forms. Word forms encode the grammatical information carried by the word and are therefore POS-specific. One of the word forms is considered the base form - or lemma of the word. All possible word forms of a word represent the lexeme of this word. From a semantic perspective, a word is assigned to a minimal sense-bearing lexical unit (LU). In general, multi-word and sub-word lexical units are possible, but in this paper we focus on single-word units. LUs are the reference point to the semantics of the word and might be used to describe further properties of the words within a specific lexicon, e.g. get assigned one or several senses, related to other LUs via lexical relations etc. Figure 1 illustrates these concepts and positions some of the relevant approaches w.r.t. the level they operate on. In general, the training objective in the unsupervised word embedding setup (e.g. (Mikolov et al., 2013)) is to induce vector representations for targets based on …

Content truncated to 500 words