UWB at SemEval-2016 Task 11: Exploring Features for Complex Word Identification

Proceedings of SemEval-2016, pages 1038–1041, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics UWB at SemEval-2016 Task 11: Exploring Features for Complex Word Identification Michal Konkol NTIS – New Technologies for the Information Society & Department of Computer Science and Engineering, Faculty of Applied Sciences, University of West Bohemia, Univerzitn´ı 8, 306 14 Plzeˇn Czech Republic konkol@kiv.zcu.cz Abstract In this paper, we present our system devel- oped for the SemEval 2016 Task 11: Com- plex Word Identification. Our team achieved the 3rd place among 21 participants. Our sys- tems ranked 4th and 13th among 42 submitted systems. We proposed multiple features suit- able for complex word identification, evalu- ated them, and discussed their properties. Ac- cording to the results of our experiments, our final system used maximum entropy classifier with a single feature – document frequency. 1 Introduction This paper describes our participation in the Com- plex Word Identification (CWI) shared task of Se- mEval 2016. CWI is a subtask of text simplification. Text simplification changes the structure, grammar, and vocabulary of the text to make it easier to under- stand without losing information. CWI looks for the words that should be simplified. The task is moti- vated and described in detail by the organizers (Paet- zold and Specia, 2016). In this paper we apply a machine learning ap- proach to CWI. Our main goal is to explore suitable features. The paper has the following structure. Section 2 introduces the task in more detail. Section 3 presents the design of our system. In Section 4, we choose the optimal parameters and features for our final system. In Section 5, we discuss overall results of the task. Section 6 summarizes our contribution. 2 Task The task is defined as a binary classification with classes complex and simple. The complex class rep- resents words that should be simplified. Given a word in a sentence, a system decides whether the word is complex or not. Paetzold and Specia (2016) prepared two train- ing data sets. The first data set contains 20 deci- sions from unique annotators; we call it all. Second data set aggregates the annotations – the word be- comes complex if at least one annotator considers it complex; we call it aggregated. Both data sets have 2 237 sentences (i.e. the all data set has 44 740 train- ing examples, the aggregated set has only 2 237). The organizers announced that the test data set has 9 200 sentences annotated by a single annotator. The task was evaluated using G-score: a harmonic mean of accuracy and recall of the complex class. Each team could submit two systems for evaluation. 3 System CWI is a binary classification task. We chose the maximum entropy classifier for our system, but we believe the choice of classifier has only a small im- pact compared with the choice of features. We propose features that may be suitable for the CWI task and motivate them in the following list. Word frequency – A ratio …

Content truncated to 500 words