Language Transfer Learning for Supervised Lexical Substitution

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 118–129, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Language Transfer Learning for Supervised Lexical Substitution Gerold Hintz and Chris Biemann Research Training Group AIPHES / FG Language Technology Computer Science Department, Technische Universität Darmstadt {hintz,biem}@lt.informatik.tu-darmstadt.de Abstract We propose a framework for lexical sub- stitution that is able to perform transfer learning across languages. Datasets for this task are available in at least three languages (English, Italian, and German). Previous work has addressed each of these tasks in isolation. In contrast, we regard the union of three shared tasks as a com- bined multilingual dataset. We show that a supervised system can be trained effec- tively, even if training and evaluation data are from different languages. Successful transfer learning between languages sug- gests that the learned model is in fact in- dependent of the underlying language. We combine state-of-the-art unsupervised fea- tures obtained from syntactic word em- beddings and distributional thesauri in a supervised delexicalized ranking system. Our system improves over state of the art in the full lexical substitution task in all three languages. 1 Introduction The lexical substitution task is defined as replac- ing a target word in a sentence context with a synonym, which does not alter the meaning of the utterance. Although this appears easy to hu- mans, automatically performing such a substitu- tion is challenging, as it implicitly addresses the problem of both determining semantically simi- lar substitutes, as well as resolving the ambiguity of polysemous words. In fact, lexical substitution was originally conceived as an extrinsic evaluation of Word Sense Disambiguation (WSD) when first proposed by McCarthy & Navigli (2007). How- ever, a system capable of replacing words by ap- propriate meaning-preserving substitutes can be utilized in downstream tasks that require para- phrasing of input text. Examples of such use cases include text simplification, text shortening, and summarization. Furthermore, lexical substitution can be regarded as an alternative to WSD in down- stream tasks requiring word disambiguation. For example, it was successfully applied in Semantic Textual Similarity (Bär et al., 2012). A given list of substitution words can be regarded as a vector representation modeling the meaning of a word in context. As opposed to WSD systems, this is not reliant on a predefined sense inventory, and there- fore does not have to deal with issues of cover- age, or sense granularity. On the other hand, per- forming lexical substitution is more complex than WSD, as a system has to both generate and rank a list of substitution candidates per instance. Over the last decade, a number of shared tasks in lexical substitution has been organized and a wide range of methods have been proposed. Al- though many approaches are in fact language- independent, most existing work is tailored to a single language and dataset. In this work, we investigate lexical substitution as a multilingual task, and report experimental results for English, German and Italian datasets. We consider a su- …

Content truncated to 500 words