What Substitutes Tell Us - Analysis of an “All-Words” Lexical Substitution Corpus

Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 540–549, Gothenburg, Sweden, April 26-30 2014. c⃝2014 Association for Computational Linguistics What Substitutes Tell Us – Analysis of an “All-Words” Lexical Substitution Corpus Gerhard Kremer Institute for Computational Linguistics University of Heidelberg, Germany kremer@cl.uni-heidelberg.de Katrin Erk Dept. of Linguistics University of Texas, Austin, U.S.A. katrin.erk@utexas.edu Sebastian Padó Institute for Natural Language Processing University of Stuttgart, Germany pado@ims.uni-stuttgart.de Stefan Thater Dept. of Computational Linguistics Saarland University, Saarbrücken, Germany stth@coli.uni-sb.de Abstract We present the first large-scale English “all- words lexical substitution” corpus. The size of the corpus provides a rich resource for investigations into word meaning. We investigate the nature of lexical substitute sets, comparing them to WordNet synsets. We find them to be consistent with, but more fine-grained than, synsets. We also identify significant differences to results for paraphrase ranking in context reported for the SEMEVAL lexical substitution data. This highlights the influence of corpus con- struction approaches on evaluation results. 1 Introduction Many, if not most, words have multiple meanings; for example, the word “bank” has a financial and a geographical sense. One common approach to deal with this lexical ambiguity is supervised word sense disambiguation, or WSD (McCarthy, 2008; Navigli, 2009), which frames the task as a lemma- level classification problem, to be solved by train- ing classifiers on samples of lemma instances that are labelled with their correct senses. This approach has its problems, however. First, it assumes a complete and consistent set of labels. WordNet, used in the majority of studies, does cover several 10,000 lemmas, but has been criti- cised for both its coverage and granularity. Second, WSD requires annotation for each sense and lemma, leading to an “annotation bottleneck”. A number of technical solutions have been suggested regard- ing the second problem (Ando and Zhang, 2005; Navigli and Ponzetto, 2012), but not for the first. In 2009, McCarthy and Navigli address both problems by proposing a fundamentally different approach, called Lexical Substitution (McCarthy and Navigli, 2009) which avoids capturing a word’s meaning by a single label. Instead, annotators are asked to list, for each instance of a word, one or more alternative words or phrases to be substituted for the target in this particular context. This setup provides a number of benefits over WSD. It al- lows characterising word meaning without using an ontology and can be obtained easily from native speakers through crowdsourcing. Work on mod- elling Lexical Substitution data has also assumed a different focus from WSD. It tends to see the predic- tion of substitutes along the lines of compositional lexical semantics, concentrating on explaining how word meaning is modulated in context (Mitchell and Lapata, 2010). There are, however, important shortcomings of the work in the Lexical Substitution paradigm. All existing datasets (McCarthy and Navigli, 2009; Sinha and Mihalcea, 2014; Biemann, 2013; Mc- Carthy et al., 2013) are either comparatively small, are “lexical sample” datasets, or both. “Lexical sample” datasets consist of sample sentences for each …

Content truncated to 500 words