Building Lexical Vector Representations from Concept Definitions

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 905–915, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics Building Lexical Vector Representations from Concept Definitions Danilo S. Carvalho and Minh Le Nguyen Japan Advanced Institute of Science and Technology 1-1 Asahidai, Nomi City, Ishikawa, Japan {danilo, nguyenml}@jaist.ac.jp Abstract The use of distributional language repre- sentations have opened new paths in solv- ing a variety of NLP problems. However, alternative approaches can take advantage of information unavailable through pure statistical means. This paper presents a method for building vector representations from meaning unit blocks called concept definitions, which are obtained by extract- ing information from a curated linguistic resource (Wiktionary). The representa- tions obtained in this way can be compared through conventional cosine similarity and are also interpretable by humans. Eval- uation was conducted in semantic simi- larity and relatedness test sets, with re- sults indicating a performance compara- ble to other methods based on single lin- guistic resource extraction. The results also indicate noticeable performance gains when combining distributional similarity scores with the ones obtained using this approach. Additionally, a discussion on the proposed method’s shortcomings is provided in the analysis of error cases. 1 Introduction Vector-based language representation schemes have gained large popularity in Natural Language Processing (NLP) research in the recent years. Their success comes from both the asserted ben- efits in several NLP tasks and from the ability to built them from unannotated textual data, widely available in the World Wide Web. The tasks bene- fiting from vector representations include Part-of- Speech (POS) tagging (dos Santos and Zadrozny, 2014), dependency parsing (Bansal et al., 2014), Named Entity Recognition (NER) (Seok et al., 2016), Machine Translation (Sutskever et al., 2014), among others. Such representation schemes are, however, not an all-in-one solution for the many NLP appli- cation scenarios. Thus, different representation methods were developed, each one focusing in a limited set of concerns, e.g., semantic relatedness measurement (Mikolov et al., 2013; Pennington et al., 2014) and grammatical dependencies (Levy and Goldberg, 2014). Most of the popular meth- ods are based on a distributional approach: the meaning of a word is defined by the context of its use, i.e., the neighboring words. However, distributional representations carry no explicit lin- guistic information and cannot easily represent some important semantic relationships, such as synonymy and antonymy (Nguyen et al., 2016). Further problems include the difficulty in obtain- ing representations for out-of-vocabulary (OOV) words and complex constructs (collocations, id- iomatic expressions), the lack of interpretable rep- resentations (Faruqui and Dyer, 2015), and the ne- cessity of specific model construction for cross- language representation. This paper presents a linguistically motivated language representation method, aimed at captur- ing and providing information unavailable on dis- tributional approaches. Our contributions are: (i) a technique for building conceptual representa- tions of linguistic elements (morphemes, words, collocations, idiomatic expressions) from a single collaborative language resource (Wiktionary 1); (ii) a method of combining said representations and …

Content truncated to 500 words