HMC at SemEval-2016 Task 11: Identifying Complex Words Using Depth-limited Decision Trees

Proceedings of SemEval-2016, pages 1034–1037, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics HMC at SemEval-2016 Task 11: Identifying Complex Words Using Depth-limited Decision Trees Maury Quijada and Julie Medero Harvey Mudd College 340 E Foothill Blvd Claremont, CA, 91711, USA {mquijada, jmedero}@hmc.edu Abstract We present two systems created for SemEval- 2016s Task 11: Complex Word Identification. Our two systems, a regression tree and de- cision tree, were trained with a word’s uni- gram and lemma word counts, average age- of-acquisition, and a measure of concreteness. The systems ranked 5th and 6th, respectively, on the test set by G-score (the harmonic mean between accuracy and recall). With the regres- sion tree’s predictions earning a G-score of 0.766, and the decision tree’s earning 0.765, the two systems scored within 1 percent of the score of the best-performing system in the task. 1 Introduction Text simplification is the process of reducing the complexity of a text while preserving the original meaning. Text simplification may include syntactic or pragmatic aspects (Siddharthan, 2014), but much of the work that has been done has focused on lex- ical simplifications. In lexical simplification, diffi- cult words or phrases are replaced to make a text more accessible. This kind of simplification can benefit several reader populations, including second- language learners (Petersen and Ostendorf, 2009). One important first step in lexical simplification is complex word identification. This step predicts which words in a text will be difficult for a reader so that they can then be targeted for simplification. The International Workshop on Semantic Evalua- tion for 2016 (SemEval-2016) hosted Task 11: Com- plex Word Identification (CWI), which asked partic- ipants to identify words that would be challenging for non-native English speakers (Paetzold and Spe- cia, 2016). Task participants were given 2,237 train- ing examples. Each example contained a word, the sentence containing the word, and the word’s index in the sentence. In addition, each word was labeled as complex or not complex (or simple) by 20 human annotators, and each word was given a binary label of complex if at least one annotator thought the word was hard to read. The individual labels of the 20 an- notators were also made available to participants. Submissions were evaluated on a test set of 88,221 words. Test set items had the same format as the training set, except that they were only anno- tated by one person, so the labels indicated whether that annotator alone labeled it complex. 2 Previous Work This is the first year of the CWI Task, so no previous work has been done on exactly the same task. How- ever, substantial work has been done previously in the general area of characterizing word difficulty. Traditional readability measures like Flesch- Kincaid (Kincaid and others, 1975) and Gunning Fog (Gunning, 1952) rely primarily on word length, while the Lexile framework considers word length and unigram frequency. More recent work has incor- porated n-grams and part of speech information (Pe- tersen and Ostendorf, 2009; Graesser …

Content truncated to 500 words