Definition Frames: Using Definitions for Hybrid Concept Representations

Proceedings of the 28th International Conference on Computational Linguistics, pages 3060–3068 Barcelona, Spain (Online), December 8-13, 2020 3060 Definition Frames: Using Definitions for Hybrid Concept Representations Evangelia Spiliopoulou Artidoro Pagnoni Eduard Hovy Language Technologies Institute Carnegie Mellon University {espiliop,apagnoni,hovy}@cs.cmu.edu Abstract Advances in word representations have shown tremendous improvements in downstream NLP tasks, but lack semantic interpretability. In this paper, we introduce Definition Frames (DF), a matrix distributed representation extracted from definitions, where each dimension is semanti- cally interpretable. DF dimensions correspond to the Qualia structure relations (Boguraev and Pustejovsky, 1990): a set of relations that uniquely define a term. Our results show that DFs have competitive performance with other distributional semantic approaches on word similarity tasks. 1 Introduction Ontologies have been widely used in lexical semantics to organize and represent knowledge. Carefully built by experts, they contain semantically meaningful information in the form of relations between concepts. However, being manually constructed, they struggle to assimilate new information. Compared to ontologies, distributed representations are fully automated and can be fine-tuned for new tasks. Despite their exceptional performance, most distributional methods do not have an explicit se- mantic interpretation. The resulting representations encode a tremendous amount of information, but afford no way to interpret what this information is and how it relates to the concept. Thus, one cannot choose which type of information is useful for a specific task, unless one has a lot of data and resources to fine-tune. Although a few approaches have tried to bridge the gap between semantics and distributed rep- resentations (Faruqui et al., 2015; Mrkˇsi´c et al., 2017), (1) they only encode information from ontologies, which are not extensible, and (2) the final representations are still not semantically meaningful. Motivated by these problems, we introduce a novel hybrid representation called Definition Frames (DF), which encode semantic information extracted from definitions. DFs are matrix representations, where each row corresponds to a particular relation. The set of the relations used is based on the Qualia structure suggested in Boguraev and Postojovsky (1990), and they are extracted automatically from definitions via a domain-adaptation approach. To the best of our knowledge, DF is the first hybrid representation, combining an explicit structure through semantically meaningful rows, while still being decomposed into distributional vectors. 2 Prior Work Prior research on lexical semantics has established a set of relations that are sufficient to uniquely define a concept. Such work includes the Qualia structure (Boguraev and Pustejovsky, 1990) and the generative lexicon theory (Pustejovsky, 1991). Other related work includes ontological approaches (Baker et al., 1998; Miller, 1995; Lenat, 1995; Speer and Havasi, 2012) and more fine-grained definition-based frames like Semagrams (Moerdijk and others, 2008). In distributional semantics, approaches including GloVe (Pennington et al., 2014), word2vec (Mikolov et al., 2013), and fastText (Bojanowski et al., 2017) obtain generic word embeddings by pre-training on large corpora. Recent work focused on context-sensitive embeddings like ELMo (Peters et al., 2018) and BERT (Devlin et al., 2018), which achieve significant improvements in downstream NLP tasks. This work is licensed …

Content truncated to 500 words