MSejrKu at SemEval-2016 Task 14: Taxonomy Enrichment by Evidence Ranking

Proceedings of SemEval-2016, pages 1337–1341, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics MSejrKu at SemEval-2016 Task 14: Taxonomy Enrichment by Evidence Ranking Michael Sejr Schlichtkrull University of Copenhagen (Denmark) michael.sejr@gmail.com H´ector Mart´ınez Alonso Univ. Paris 7 - INRIA (France) hector.martinez-alonso@inria.fr Abstract Automatic enrichment of semantic tax- onomies with novel data is a relatively unexplored task with potential benefits in a broad array of natural language processing problems. Task 14 of SemEval 2016 poses the challenge of designing systems for this task. In this paper, we describe and evaluate several machine learning systems constructed for our participation in the competition. We demonstrate an f1-score of 0.680 for our submitted systems — a small improvement over the 0.679 produced by the hard baseline. 1 Introduction This article describes our systems submitted for Se- mEval2016, task 14 on taxonomy enrichment. 1 The two submitted runs are based on Gaussian-kernel SVMs, and fall respectively under the constrained and the unconstrained condition of the shared task, namely using only WordNet and the training data, or incorporating outside sources. We chose our submit- ted models by evaluation on the trial data in lieu of a development set. Our runs perform better than the very hard baseline with a small 2% margin, while the best-performing heuristic outperforms the base- line by a 5% margin. 2 Related Work Existing methods for taxonomy enrichment can roughly be divided into two categories: relying on alignment between multiple taxonomies, or relying on machine learning-based rating of subgraphs. 1http://alt.qcri.org/semeval2016/task14/ In (Jurgens and Pilehvar, 2015), Wordnet is ex- tended with technical terms and rare lemmas from Wiktionary. In (Suchanek et al., 2007), relation- extraction is used to unify WordNet and Wikipedia. (Navigli and Ponzetto, 2012) further automates the alignment process itself. In (Toral et al., 2008), named entities are brought from Wikipedia to Word- Net through pattern matching. In (Snow et al., 2006), the probabilities of tax- onomies are evaluated based on evidence vectors as- sociated to edges. A similar approach formulated in terms of factor graphs can be seen in (Bansal et al., 2013). Finally, (Yamada et al., 2011) employ a hybrid strategy, scoring edges by likelihood of ap- pearance in Wikipedia. Our approach also falls in the second category, relying on a machine learning process to rank hypernym-hyponym edges. 3 Data There are three data splits: train, trial, and test, de- scribed in Table 1. We use the trial data for model selection. Notice that attach-actions represent the vast majority of instances. We therefore focus ex- clusively on finding correct integrations and disre- gard the merge-attach distinction. Several observations can be made from the distri- bution of the labels in the training set. First, we de- fine the closest synset sc(t, g) of a term t with gold standard g as the synset such that a lemma of that synset is represented in some description sentence d ∈ D(t) and the shortest path δ(g, sc(t, g)) in the taxonomy is minimized. Here, D(t) is the set of …

Content truncated to 500 words