EELECTION at SemEval-2017 Task 10: Ensemble of nEural Learners for kEyphrase ClassificaTION

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 942–946, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics EELECTION at SemEval-2017 Task 10: Ensemble of nEural Learners for kEyphrase ClassificaTION Steffen Eger†‡, Erik-Lˆan Do Dinh†, Ilia Kuznetsov†, Masoud Kiaeeha†, Iryna Gurevych†‡ †Ubiquitous Knowledge Processing Lab (UKP-TUDA) Department of Computer Science, Technische Universit¨at Darmstadt ‡Ubiquitous Knowledge Processing Lab (UKP-DIPF) German Institute for Educational Research and Educational Information http://www.ukp.tu-darmstadt.de Abstract This paper describes our approach to the SemEval 2017 Task 10: “Extracting Keyphrases and Relations from Scientific Publications”, specifically to Subtask (B): “Classification of identified keyphrases”. We explored three different deep learn- ing approaches: a character-level convo- lutional neural network (CNN), a stacked learner with an MLP meta-classifier, and an attention based Bi-LSTM. From these approaches, we created an ensemble of differently hyper-parameterized systems, achieving a micro-F1-score of 0.63 on the test data. Our approach ranks 2nd (score of 1st placed system: 0.64) out of four ac- cording to this official score. However, we erroneously trained 2 out of 3 neural nets (the stacker and the CNN) on only roughly 15% of the full data, namely, the origi- nal development set. When trained on the full data (training+development), our en- semble has a micro-F1-score of 0.69. Our code is available from https://github. com/UKPLab/semeval2017-scienceie. 1 Introduction Although scientific experiments are often accom- panied by vast amounts of structured data, full-text scientific publications still remain one of the main means for communicating academic knowledge. Given the dynamic nature of modern research and its ever-accelerating pace, it is crucial to automat- ically analyze new works in order to have a com- plete picture of advances in a given field. Recently, some progress has been made in this direction for the fixed-domain use case1. How- ever, creating a universal open-domain system still 1 E.g. BioNLP: http://2016.bionlp-st.org/ remains a challenge due to significant domain dif- ferences between articles originating from differ- ent fields of research. The SemEval 2017 Task 10: ScienceIE (Augenstein et al., 2017) promotes the multi-domain use case, providing source ar- ticles from three domains: Computer Science, Material Sciences and Physics. The task con- sists of three subtasks, namely (A) identification of keyphrases, (B) classifying them into broad domain-independent classes and (C) inferring re- lations between the identified keyphrases. For example, for the input sentence ‘The ther- modynamics of copper-zinc alloys (brass) was subject of numerous investigations’ the following output would be expected: (A) 1. The thermodynamics of copper-zinc alloys 2. copper-zinc alloys 3. brass (B) 1. TASK 2. MATERIAL 3. MATERIAL (C) synonym(2,3) Our submission focuses on (B) keyphrase clas- sification given item boundaries. We avoid task- specific feature engineering, which would poten- tially render the system domain-dependent. In- stead, we build an ensemble of several deep learn- ing classifiers detailed in §3, whose inputs are word embeddings learned from general domains. 2 Task and Data In the annotation scheme proposed by the task or- ganizers, keyphrases denoting a scientific model, algorithm or process should be classified as …

Content truncated to 500 words