SZTE-NLP at SemEval-2017 Task 10: A High Precision Sequence Model for Keyphrase Extraction Utilizing Sparse Coding for Feature Generation

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 990–994, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics SZTE-NLP at SemEval-2017 Task 10: A High Precision Sequence Model for Keyphrase Extraction Utilizing Sparse Coding for Feature Generation G´abor Berend Department of Informatics, University of Szeged ´Arp´ad t´er 2, H6720 Szeged, Hungary berendg@inf.u-szeged.hu Abstract In this paper we introduce our system participating at the 2017 SemEval shared task on keyphrase extraction from scien- tific documents. We aimed at the creation of a keyphrase extraction approach which relies on as little external resources as pos- sible. Without applying any hand-crafted external resources, and only utilizing a transformed version of word embeddings trained at Wikipedia, our proposed system manages to perform among the best partic- ipating systems in terms of precision. 1 Introduction The sheer amount of scientific publications makes intelligent processing of papers increasingly im- portant. Automated keyphrase extraction tech- niques can mitigate the severe difficulties arising when navigating in massive document collections. Hence, extracting keyphrases from scientific lit- erature has generated substantial academic inter- est over the past years (Witten et al., 1999; Hulth, 2003; Kim et al., 2010; Berend, 2016a). Continuous word representations such as word2vec (Mikolov et al., 2013) has gained increasing popularity recently. These representa- tions assign some semantically meaningful low dimensional vector wi to the vocabulary entries of large text corpora. We demonstrated previously (Berend, 2016b) that useful features can be derived for various se- quence labeling tasks by performing a sparse de- composition of the word embedding matrix. In this paper, we investigate the generalization prop- erties of our proposed approach for the task of keyphrase extraction. 2 Sequence labeling framework Our sequence labeling framework builds on top of our previous work which aimed at multiple dif- ferent sequence labeling tasks, i.e. part-of-speech tagging and named entity recognition. 2.1 Feature representation Each token in a sequence is described by a set of feature values and those of its direct neighbors in our model. We relied on multiple sources for de- riving features, i.e. • sparse coding of dense word embeddings, • Brown clustering of words, • word identity features and • orthographic characteristics. 2.1.1 Sparse coding derived features The main source of features was sparse coding performed on continuous word embeddings. We demonstrated in (Berend, 2016b) that sequence la- beling tasks can largely benefit from the sparse decomposition of dense word embedding matri- ces. That is, given a word embedding matrix W ∈ Rd×|V | – with its columns containing the d dimensional dense word embeddings – we seek for its decomposition into a product of D ∈ Rd×K and α ∈ RK×|V | – containing sparse linear com- bination coefficients for each of the word embed- dings – such that ∥W − Dα∥2 F + λ∥α∥1 gets min- imized. Features for some word wi are then determined based on its corresponding vector αi by taking the signs and indices of its non-zero coefficients, i.e. f(wi) = {sign(αi[j])j | αi[j] …

Content truncated to 500 words