VCU at Semeval-2016 Task 14: Evaluating definitional-based similarity measure for semantic taxonomy enrichment

Proceedings of SemEval-2016, pages 1351–1355, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics VCU at Semeval-2016 Task 14: Evaluating similarity measures for semantic taxonomy enrichment Bridget T. McInnes Department of Computer Science Virginia Commonwealth University Richmond, VA 23284 btmcinnes@vcu.edu Abstract This paper describes the VCU systems that participated in the Semantic Taxonomy En- richment task of SemEval 2016. The three systems are unsupervised and relied on dictionary-based similarity measures. The first two runs used first-order measures (Lesk and First-order vector), and the third run used a second-order measure (Second-order vec- tor). The first-order measures obtained a higher Wu & Palmer score than the second- order measure on the test data. All three runs obtained a higher Wu & Palmer and F1 score than the random baseline but not the first-word first-sense baseline. 1 Introduction Semantic knowledge bases are used in a number of NLP applications, e.g. Word Sense Disambigua- tion ((Agirre et al., 2014) (Agirre et al., 2010)) and Information Retrieval (Uddin et al., 2013). These knowledge bases span across a number of domains, e.g. WordNet, Gene Ontology, and Medical Sub- ject Headings. Building and maintaining these knowledge bases is expensive and manually inten- sive (Martinez-Gil, 2015). Semantic taxonomy en- richment aims to aid in the maintenance process by automatically placing new terms into an existing tax- onomy. The Semantic Taxonomy Enrichment task (Se- mEval 2016 Task 14) objective was to automatically incorporate new word senses into WordNet 1, a lex- ical dictionary of English terms that are linked to- gether based on a number of relations (e.g. is-a). 1https://wordnet.princeton.edu/ In this task, systems were provided a list of out- of-vocabulary terms (OOVs), their part-of-speech (POS), and a brief description of the term (OOV description). The goal of this task was to auto- matically identify which WordNet synset the OOV should be associated with and whether the associa- tion was synonymous (merge) or the OOV is a hy- ponym of the synset (attach). Semantic similarity measures have been shown useful in the development of terminologies and on- tologies (Vizenor et al., 2009). These measures quantify how related two terms are. The measures that we focus on for this task rely on definitional in- formation extracted from knowledge sources such as WordNet. Our approach to the shared task is a modification of these measures relying on Word- Net’s gloss information, and the OOV descriptions. We categorize the measures we evaluated as first- order and second-order measures. The first-order measures conduct a direct comparison between the words in the WordNet synset’s gloss and the words in the OOV description. Second-order measures in- corporate additional context by creating a vector for each word in the synset’s gloss (or OOV description) containing words that co-occur with it from an exter- nal corpus. These word vectors are then averaged to create a single co-occurrence vector for the OOV or synset. The VCU systems were implemented using the freely available, open source UMLS-Similarity package (McInnes et al., 2009) (version 1.45), which includes support …

Content truncated to 500 words