NTNU-1@ScienceIE at SemEval-2017 Task 10: Identifying and Labelling Keyphrases with Conditional Random Fields

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 938–941, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics NTNU-1@ScienceIE at SemEval-2017 Task 10: Identifying and Labelling Keyphrases with Conditional Random Fields Erwin Marsi, Utpal Skidar, Cristina Marco, Biswanath Barik, Rune Saetre Department of Computer Science Norwegian University of Science and Technology {emarsi,utpals,cristina.marco,biswanath.barik,saetre}@ntnu.no Abstract We present NTNU’s systems for Task A (prediction of keyphrases) and Task B (labelling as Material, Process or Task) at SemEval 2017 Task 10: Extracting Keyphrases and Relations from Scienti- fic Publications (Augenstein et al., 2017). Our approach relies on supervised ma- chine learning using Conditional Random Fields. Our system yields a micro F-score of 0.34 for Tasks A and B combined on the test data. For Task C (relation extrac- tion), we relied on an independently de- veloped system described in (Barik and Marsi, 2017). For the full Scenario 1 (in- cluding relations), our approach reaches a micro F-score of 0.33 (5th place). Here we describe our systems, report results and discuss errors. 1 Approach We choose Conditional Random Fields (Lafferty et al., 2001) because they have produced state-of- art results on comparable sequence labelling tasks such as named entity recognition in biomedicine. Two systems were developed, using different fea- ture sets and alternative CRF implementations. Preprocessing Input text is linguistically anal- ysed using the Spacy NLP pipeline (Honnibal and Johnson, 2015), including sentence splitting, to- kenisation, lemmatisation and dependency pars- ing. Since CRFs cannot handle Brat’s stand-off annotation format directly, keyphrase annotations are first converted to the Inside-Outside-Begin (IOB) tagging scheme by aligning their character offsets with the character offsets of tokens: if the start character offset of a token coincides with the start offset of an annotated keyphrase, the token receives a B (begin) tag; if the offset span of a to- ken falls within the offset bounds of a keyphrase, the token gets an I (inside) tag, otherwise the token is assigned an O (outside) tag. Each sentence cor- responds to a sequence of IOB tags, serving as the labelled sequence for the CRF. Separate IOB tags are derived for each of the three keyphrase classes (Material, Process, Task). Annotations and tokens do not always properly align; the resulting errors are discussed in Section 3. System 1 relies on the CRFsuite implementa- tion (Okazaki, 2007) as wrapped by the sklearn- crfsuite module for SciKit Learn. A dedicated classifier is trained for each of the three keyphrase classes. CRFs are used with default parameter setting. The following features were selected per class by cross-validation on the development data: • Word features: word shape (e.g. ’Xxxx’), is- alpha, is-lower-case, is-ascii, is-capitalized, is-upper-case, is-punctuation, like-number, prefix-chars (2,3,4), suffix-chars (2,3,4), is- stopword, all in a window of size 3 for Mate- rial and Process; • Lemma and POS, in a window of size 5 for Material and Process, in window of size 3 for Process; • Wordnet (for Material only): synset names of all hypernyms (transitive closure), in a win- dow …

Content truncated to 500 words