SemEval-2013 Task 4: Free Paraphrases of Noun Compounds

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 138–143, Atlanta, Georgia, June 14-15, 2013. c⃝2013 Association for Computational Linguistics SemEval-2013 Task 4: Free Paraphrases of Noun Compounds Iris Hendrickx Radboud University Nijmegen & Universidade de Lisboa iris@clul.ul.pt Preslav Nakov QCRI, Qatar Foundation pnakov@qf.org.qa Stan Szpakowicz University of Ottawa & Polish Academy of Sciences szpak@eecs.uottawa.ca Zornitsa Kozareva University of Southern California kozareva@isi.edu Diarmuid ´O S´eaghdha University of Cambridge do242@cam.ac.uk Tony Veale University College Dublin tony.veale@ucd.ie Abstract In this paper, we describe SemEval-2013 Task 4: the definition, the data, the evaluation and the results. The task is to capture some of the meaning of English noun compounds via para- phrasing. Given a two-word noun compound, the participating system is asked to produce an explicitly ranked list of its free-form para- phrases. The list is automatically compared and evaluated against a similarly ranked list of paraphrases proposed by human annota- tors, recruited and managed through Ama- zon’s Mechanical Turk. The comparison of raw paraphrases is sensitive to syntactic and morphological variation. The “gold” ranking is based on the relative popularity of para- phrases among annotators. To make the rank- ing more reliable, highly similar paraphrases are grouped, so as to downplay superficial dif- ferences in syntax and morphology. Three systems participated in the task. They all beat a simple baseline on one of the two evalua- tion measures, but not on both measures. This shows that the task is difficult. 1 Introduction A noun compound (NC) is a sequence of nouns which act as a single noun (Downing, 1977), as in these examples: colon cancer, suppressor protein, tumor suppressor protein, colon cancer tumor sup- pressor protein, etc. This type of compounding is highly productive in English. NCs comprise 3.9% and 2.6% of all tokens in the Reuters corpus and the British National Corpus (BNC), respectively (Bald- win and Tanaka, 2004). The frequency spectrum of compound types fol- lows a Zipfian distribution ( ´O S´eaghdha, 2008), so many NC tokens belong to a “long tail” of low- frequency types. More than half of the two-noun types in the BNC occur exactly once (Kim and Bald- win, 2006). Their high frequency and high produc- tivity make robust NC interpretation an important goal for broad-coverage semantic processing of En- glish texts. Systems which ignore NCs may give up on salient information about the semantic relation- ships implicit in a text. Compositional interpretation is also the only way to achieve broad NC coverage, because it is not feasible to list in a lexicon all com- pounds which one is likely to encounter. Even for relatively frequent NCs occurring 10 times or more in the BNC, static English dictionaries provide only 27% coverage (Tanaka and Baldwin, 2003). In many natural language processing applications it is important to understand the syntax and seman- tics of NCs. NCs often are structurally similar, but have very different meaning. Consider caffeine headache and ice-cream headache: a lack of caf- feine …

Content truncated to 500 words