TALN at SemEval-2016 Task 11: Modelling Complex Words by Contextual, Lexical and Semantic Features

Proceedings of SemEval-2016, pages 1011–1016, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics TALN at SemEval-2016 Task 11: Modelling Complex Words by Contextual, Lexical and Semantic Features Francesco Ronzano, Ahmed Abura’ed, Luis Espinosa-Anke, Horacio Saggion Department of Information and Communication Technologies (DTIC) Universitat Pompeu Fabra Tang`er 122, Barcelona (08018), Spain {francesco.ronzano,ahmed.aburaed,luis.espinosa,horacio.saggion}@upf.edu Abstract This paper presents the participation of the TALN team in the Complex Word Identifica- tion Task of SemEval-2016 (Task 11). The purpose of the task was to determine if a word in a given sentence can be judged as com- plex or not by a certain target audience. To experiment with word complexity identifica- tion approaches, Task organizers provided a training set of 2,237 words judged as com- plex or not by 20 human evaluators, together with the sentence in which each word occurs. In our contribution we modelled each word to evaluate as a numeric vector populated with a set of lexical, semantic and contextual fea- tures that may help assess the complexity of a word. We trained a Random Forest clas- sifier to automatically decide if each word is complex or not. We submitted two runs in which we respectively considered unweighted and weighted instances of complex words to train our classifier, where the weight of each instance is proportional to the number of eval- uators that judged the word as complex. Our system scored as the third best performing one. 1 Introduction Approaches to automatically identify if a target au- dience will perceive a certain word as complex or not constitute a core component in several language- related areas of research, including Lexical Simpli- fication (Bott et al., 2012) and Readability Assess- ment (Collins-Thompson, 2014). The Complex Word Identification Task of SemEval-2016 proposes a shared framework for evaluating complex word identification systems. Task participants were provided with a set of sen- tences where, for each sentence, one or more words have been rated as complex or not by 20 human eval- uators. An example sentence from this dataset is: If the growth rate is known , the maximum lichen size will give a minimum age for when this rock was deposited. In this sentence, the words ’lichen’ and ’de- posited’ were classified as complex by at least one out of the 20 evaluators, unlike e.g. ’growth’, which did not received this label by any of them. In our participation in Task 11, we cast the identi- fication of complex words as a binary classification problem in which each word is evaluated as com- plex or not, given the sentence in which it occurs. We modelled each word by a set of lexical, semantic and contextual features and evaluated distinct binary classification algorithms. Our approach to Task 11 obtained good performance: our team ranked as the second best performing one and one of the two sys- tems we proposed scored as the third best perform- ing system according to the G-score official evalua- tion metric (harmonic mean between Accuracy and Recall). In Section …

Content truncated to 500 words