MAZA at SemEval-2016 Task 11: Detecting Lexical Complexity Using a Decision Stump Meta-Classifier

Proceedings of SemEval-2016, pages 991–995, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics MAZA at SemEval-2016 Task 11: Detecting Lexical Complexity Using a Decision Stump Meta-Classifier Shervin Malmasi Macquarie University Sydney, NSW, Australia shervin.malmasi@mq.edu.au Marcos Zampieri Saarland Unviersity Saarbr¨ucken, Germany marcos.zampieri@dfki.de Abstract This paper describes team MAZA entries for the 2016 SemEval Task 11: Complex Word Identification (CWI). The task is a binary clas- sification task in which systems are trained to predict whether a word in a sentence is con- sidered to be complex or not. We developed our two systems for this task based on classi- fier stacking using decision stumps and deci- sion trees. Our best system, using contextual features, frequency information, and word and sentence length, achieved 91.2% accuracy and 30.8% F-Score. The system ranked 4th among the 38 entries in the CWI task in terms of F- Score. 1 Introduction Lexical simplification is a popular task in natural language processing and it was the topic of a suc- cessful SemEval task in 2012 (Specia et al., 2012). It consists of applying computational methods to sub- stitute words or short phrases for simpler ones to improve text readability and comprehension aimed at a given target population (e.g. children, lan- guage learners, people with reading impairment, etc.). Lexical simplification is considered to be the sub-task of text simplification that deals with the lexicon while other sub-tasks address, for example, complex syntactic structures (Siddharthan, 2014). To perform lexical simplification efficiently, com- putational methods should be first applied to identify which words in a text pose more difficulty to readers and they therefore good candidates for substitution (Shardlow, 2013). This task is called complex word identification (CWI) and it is the topic of the 2016 SemEval Task 11 with the same name. The CWI shared task is modeled as a binary text classification task. Participants are provided with training data containing sentences and a label for each word in them containing a value of either 1 (for complex words) or 0 (for simple words). The la- bel was attributed according to the judgment of hu- man annotators that were required to indicate which words in the sentences could not be easily under- stood. Below, an example can be found of a sentence from the training set. Complex words are marked in bold. (1) The name ‘kangaroo mouse’ refers to the species’ extraordinary jumping ability, as well as its habit of bipedal locomotion. In the example presented above, the CWI systems should label extraordinary, bipedal and locomotion as complex words.1 To accomplish this task, the MAZA team applied a decision stump meta-classifier and a wide set of features that we will describe here. 2 Data Organizers of the SemEval CWI task provided a training and test set comprising English sentences with each word annotated with a complex or simple label. According to the CWI task website2: ‘400 an- notators were presented with several sentences and asked to select which words they did not understand 1Note that participants …

Content truncated to 500 words