Supervised All-Words Lexical Substitution using Delexicalized Features

Proceedings of NAACL-HLT 2013, pages 1131–1141, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Supervised All-Words Lexical Substitution using Delexicalized Features Gy¨orgy Szarvas1 Chris Biemann2 Iryna Gurevych3,4 (1) Nuance Communications Deutschland GmbH Kackertstrasse 10, D-52072 Aachen, Germany (2) FG Language Technology Department of Computer Science, Technische Universit¨at Darmstadt (3) Ubiquitous Knowledge Processing Lab (UKP-TUDA) Department of Computer Science, Technische Universit¨at Darmstadt (4) Ubiquitous Knowledge Processing Lab (UKP-DIPF) German Institute for Educational Research and Educational Information http://www.nuance.com , http://www.ukp.tu-darmstadt.de Abstract We propose a supervised lexical substitu- tion system that does not use separate clas- sifiers per word and is therefore applicable to any word in the vocabulary. Instead of learning word-specific substitution patterns, a global model for lexical substitution is trained on delexicalized (i.e., non lexical) features, which allows to exploit the power of super- vised methods while being able to general- ize beyond target words in the training set. This way, our approach remains technically straightforward, provides better performance and similar coverage in comparison to unsu- pervised approaches. Using features from lex- ical resources, as well as a variety of features computed from large corpora (n-gram counts, distributional similarity) and a ranking method based on the posterior probabilities obtained from a Maximum Entropy classifier, we im- prove over the state of the art in the LexSub Best-Precision metric and the Generalized Av- erage Precision measure. Robustness of our approach is demonstrated by evaluating it suc- cessfully on two different datasets. 1 Introduction In recent years, the task of automatically providing lexical substitutions in context (McCarthy and Nav- igli, 2007) received much attention. The premise to be able to replace words in a sentence with- out changing its meaning gave rise to applications like linguistic steganography (Topkara et al., 2006; Chang and Clark, 2010), semantic text similarity (Agirre et al., 2012), and plagiarism detection (Gipp et al., 2011). Lexical substitution, a special form of contex- tual paraphrasing where only a single word is re- placed, is closely related to word sense disambigua- tion (WSD): polysemous words have possible sub- stitutions reflecting several senses, and the correct sense has to be picked to avoid spurious system be- havior. However, no explicit word sense inventory is required for lexical substitution (Dagan et al., 2006). The prominent tasks in a lexical substitution sys- tem are generation and ranking, i.e. to generate a set of possible substitutions for the target word and then to rank this set of possible substitutions according to their contextual fitness. The task to generate a high quality set of possible substitutions is challenging in itself, for two reasons. First, the available lexical resources are seldom complete in listing synonyms. Second, manually annotated substitutions show that not all synonyms of a word are appropriate in a given context, and many good substitutions have other lex- ical relation than synonymy to the original word. In this work, we present a supervised lexical sub- stitution system that, unlike the usual lexical sam- ple supervised approaches, can produce substitu- tions for targets …

Content truncated to 500 words