DeftPunk at SemEval-2020 Task 6: Using RNN-ensemble for the Sentence Classification.

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 697–703 Barcelona, Spain (Online), December 12, 2020. 697 DeftPunk at SemEval-2020 Task 6: using RNN-ensemble for the sentence classification Jekaterina Kaparina jekaterina.kaparina@student.uni-tuebingen.de Anna Soboleva anna.soboleva@student.uni-tuebingen.de Abstract This paper describes participation in DeftEval 2020 (part of SemEval sharing task competition), and is focused on the sentence classification. Our approach to the task was to create an ensemble of several RNNs combined with fasttext and ELMo embeddings. Results show that various types of models in an ensemble give a performance boost in comparison to standard models. Our model achieved F1-score of 78% for a positive class on the DeftEval dataset. 1 Introduction Definition extraction has been a popular topic in NLP for a long time. Early research in this field dates back to the end of the 20th century (Nakamura and Nagao, 1988). Throughout the last two decades numerous methods to solve the problem were proposed: rule-based approaches (Storrer and Wellinghoff, 2006; Pollak et al., 2012), machine learning approaches (for example, genetic algorithms (Borg et al., 2009), probabilistic topic models (Faralli and Navigli, 2013)) etc. Interest in the definition extraction can be explained by the fact that it is applicable in a large number of areas, including e-learning (Monachesi and Westerhout, 2008), glossary creation (Park et al., 2002), Word Sense Disambiguation (Camacho-Collados et al., 2015) and Information Extraction (Delli Bovi et al., 2015). The goal of the task was to correctly label sentences as definitions or non-definitions. According to the organizers, most of the research in the definition extraction was based on the assumption that definitions can be retrieved by detecting keywords or syntactic structure, following the so-called Aristotelian idea of the definition: "X is Y, which is..." (Spala et al., 2019). The corpus, used for this task, is focused on other types of definitions which are harder to capture with simple pattern detection. Spala et al. (2019) summed up two main approaches to classification of definitions: usually the main method was either pattern matching or hypernym detection. They also noted that previous research was aimed primarily at English sources and that the number of available definition extraction corpora is quite limited. Perhaps because the first research in the field was conducted relatively long ago and the open source culture was not as strong, a large number of corpora remained private. This is especially visible in Espinosa-Anke et al. (2015), where seven Definition Extraction corpora are listed, but only two of them are easily accessible. Apart from the DEFT corpus, used in this shared task, to our knowledge, only two other definition extraction corpora are publicly available: so-called W00 corpus (Jin et al., 2013) and WCL corpus (Navigli et al., 2010). Unfortunately, their material mostly follows the Aristotelian model of definition, hence, they were ineffectual as additional training data. It is noticeable that the problem of definition extraction has become less fundamental in recent years. For example, the state-of-the-art model, described in Espinosa-Anke and Schockaert (2018) is one of the few examples …

Content truncated to 500 words