NTNU at SemEval-2018 Task 7: Classifier Ensembling for Semantic Relation Identification and Classification in Scientific Papers

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 858–862 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics NTNU at SemEval-2018 Task 7: Classifier Ensembling for Semantic Relation Identification and Classification in Scientific Papers Biswanath Barik1, Utpal Kumar Sikdar2 and Bj¨orn Gamb¨ack1 1Department of Computer Science, NTNU, Norway 2Flytxt, Thiruvananthapuram, India {biswanath.barik,gamback}@ntnu.no utpal.sikdar@gmail.com Abstract The paper presents NTNU’s contribution to SemEval-2018 Task 7 on relation identifica- tion and classification. The class weights and parameters of five alternative supervised clas- sifiers were optimized through grid search and cross-validation. The outputs of the classi- fiers were combined through voting for the fi- nal prediction. A wide variety of features were explored, with the most informative identified by feature selection. The best setting achieved F1 scores of 47.4% and 66.0% in the relation classification subtasks 1.1 and 1.2. For relation identification and classification in subtask 2, it achieved F1 scores of 33.9% and 17.0%, 1 Introduction Scientific papers are valuable knowledge sources providing authentic insights about certain aspects of the research domains. With the advancement of scientific research, a massive growth of published articles are observed. As per the American Journal Experts (AJE) scholarly publishing report1, ap- proximately 2.2 million articles were added to the literature in 2016 only. The sheer volume of the ever increasing literature of any scientific disci- pline makes it hard for human capability and ex- pertise to quickly process and identify information of interest. Therefore, there is a need to efficiently exploit automatic means of accessing this reliable unstructured knowledge repository. Semantic relation extraction is one of the main information extraction tasks, and aims to identify a pair of arguments connected by certain prede- fined relation types based on a target application. The relation arguments are of different types such as Named Entities (Freitas et al., 2009), nomi- nals (Hendrickx et al., 2009), general keyphrases 1https://www.aje.com/en/arc/dist/docs/International- scholarly-publishing-report-2016.pdf (G´abor et al., 2016; Augenstein et al., 2017), quan- titative variables (Marsi et al., 2014) or events (Barik et al., 2017), and are syntactically repre- sented by noun phrases, clauses or larger complex structures. A semantic relation may be either sym- metric (undirected) or asymmetric (hierarchical). Supervised machine learning approaches have been successfully used for identifying semantic relations encoded in texts. Broadly, three types of supervised approaches to relation extraction have been investigated: feature-based (Kamb- hatla, 2004; Jiang and Zhai, 2007), kernel-based (Zelenko et al., 2003), and neural network based (Zeng et al., 2014; Miwa and Bansal, 2016). In this work, various relation identification and classification subtasks of SemEval 2018 Task 7 (G´abor et al., 2018) were addressed using feature- based approaches. A wide variety of features was explored, including lexical (e.g., bag-of- words, lemmata, n-grams), syntactic (e.g., part-of- speech, parsing information), semantic (e.g., de- pendency information, WordNet (Miller, 1995)), and other binary indicators. A χ2-based feature selection technique was used to identify infor- mative features. The class weights and parame- ters of five different classifiers—Support Vector Machines (SVM), Decision Trees (DT), Random Forests (RF), Multinomial …

Content truncated to 500 words