SemEval-2012 Task 1: English Lexical Simplification

First Joint Conference on Lexical and Computational Semantics (*SEM), pages 347–355, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics SemEval-2012 Task 1: English Lexical Simplification Lucia Specia Department of Computer Science University of Sheffield L.Specia@sheffield.ac.uk Sujay Kumar Jauhar Research Group in Computational Linguistics University of Wolverhampton Sujay.KumarJauhar@wlv.ac.uk Rada Mihalcea Department of Computer Science and Engineering University of North Texas rada@cs.unt.edu Abstract We describe the English Lexical Simplifica- tion task at SemEval-2012. This is the first time such a shared task has been organized and its goal is to provide a framework for the evaluation of systems for lexical simplification and foster research on context-aware lexical simplification approaches. The task requires that annotators and systems rank a number of alternative substitutes – all deemed adequate – for a target word in context, according to how “simple” these substitutes are. The notion of simplicity is biased towards non-native speak- ers of English. Out of nine participating sys- tems, the best scoring ones combine context- dependent and context-independent informa- tion, with the strongest individual contribution given by the frequency of the substitute re- gardless of its context. 1 Introduction Lexical Simplification is a subtask of Text Simpli- fication (Siddharthan, 2006) concerned with replac- ing words or short phrases by simpler variants in a context aware fashion (generally synonyms), which can be understood by a wider range of readers. It generally envisages a certain human target audience that may find it difficult or impossible to understand complex words or phrases, e.g., children, people with poor literacy levels or cognitive disabilities, or second language learners. It is similar in many re- spects to the task of Lexical Substitution (McCarthy and Navigli, 2007) in that it involves determining adequate substitutes in context, but in this case on the basis of a predefined criterion: simplicity. A common pipeline for a Lexical Simplification system includes at least three major components: (i) complexity analysis: selection of words or phrases in a text that are considered complex for the reader and/or task at hand; (ii) substitute lookup: search for adequate replacement words or phrases deemed complex in context, e.g., taking synonyms (with the same sense) from a thesaurus or finding similar words/phrases in a corpus using distributional simi- larity metrics; and (iii) context-based ranking: rank- ing of substitutes according to how simple they are to the reader/task at hand. As an example take the sentence: “Hitler com- mitted terrible atrocities during the second World War.” The system would first identify complex words, e.g. atrocities, then search for substitutes that might adequately replace it. A thesaurus lookup would yield the following synonyms: abomination, cruelty, enormity and violation, but enormity should be dropped as it does not fit the context appropri- ately. Finally, the system would determine the sim- plest of these substitutes, e.g., cruelty, and use it to replace the complex word, yielding the sentence: “Hitler committed terrible cruelties during the sec- ond World War.”. Different from other subtasks of Text Simplifica- tion like Syntactic Simplification, which have been …

Content truncated to 500 words