Know-Center at SemEval-2017 Task 10: Sequence Classification with the CODE Annotator

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 961–964, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics Know-Center at SemEval-2017 Task 10: Sequence Classification with the CODE Annotator Roman Kern Know-Center GmbH Inffeldgasse 13 Graz, 8010, Austria rkern@know-center.at Stefan Falk Know-Center GmbH Inffeldgasse 13 Graz, 8010, Austria sfalk@know-center.at Andi Rexha Know-Center GmbH Inffeldgasse 13 Graz, 8010, Austria arexha@know-center.at Abstract This paper describes our participation in SemEval-2017 Task 10, named ScienceIE (Machine Reading for Scientist). We com- peted in Subtask 1 and 2 which consist re- spectively in identifying all the key phrases in scientific publications and label them with one of the three categories: Task, Process, and Material. These scientific publications are selected from Computer Science, Mate- rial Sciences, and Physics domains. We fol- lowed a supervised approach for both subtasks by using a sequential classifier (CRF - Con- ditional Random Fields). For generating our solution we used a web-based application im- plemented in the EU-funded research project, named CODE. Our system achieved an F1 score of 0.39 for the Subtask 1 and 0.28 for the Subtask 2. 1 Introduction Information Retrieval (IR) systems for scientific publications face different challenges compared to the standard approaches. This, mainly is due to the unavailability of the whole text from reviewed pa- pers and the vague specification of the searching in- formation. The identification and the extraction of the key phrases from such articles can partially over- come the limits described above by allowing search engines to access and use them as text features. Fur- thermore, the classification of the key phrases as a Task, a Process, or a Material, can help the re- searchers to correctly specify the type of information they are seeking. Figure 1: Example of a keyphrase with its associated label The Subtasks 1 and 2 of Task 10 (Augenstein et al., 2017) in SemEval-2017 named ScienceIE (Machine Reading for Scientist), tackle the afore- mentioned problems. This task consists in identi- fying (Subtask 1) and labeling (Subtask 2) all the key phrases in scientific publications from Computer Science, Material Science, and Physics. For training and evaluating this task, it was pro- vided a set of scientific papers together with the an- notated key phrases and their associated labels. The annotations were represented with their start and end offsets in the text. The labels associated with each annotation can be from one of the three options: Task, Process, and Material. The example in Figure 1 illustrates the given dataset. We followed a supervised approach for both sub- tasks. More specifically, we trained a sequential classifier CRF - Conditional Random Fields (Laf- ferty et al., 2001) and fed it with grammatical and text features. The model built from this classifier represents our solution for identifying and labeling the key phrases. The rest of the paper is organized as follows. In the next section we describe our system’s details. In section 3 we show the results of our systems and compare it with the other …

Content truncated to 500 words