Single Document Keyphrase Extraction Using Sentence Clustering and Latent Dirichlet Allocation

Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010, pages 154–157, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics Task 5: Single document keyphrase extraction using sentence clustering and Latent Dirichlet Allocation Claude Pasquier Institute of Developmental Biology & Cancer University of Nice Sophia-Antipolis UNSA/CNRS UMR-6543 Parc Valrose 06108 NICE Cedex 2, France claude.pasquier@unice.fr Abstract This paper describes the design of a sys- tem for extracting keyphrases from a sin- gle document The principle of the algo- rithm is to cluster sentences of the doc- uments in order to highlight parts of text that are semantically related. The clusters of sentences, that reflect the themes of the document, are then analyzed to find the main topics of the text. Finally, the most important words, or groups of words, from these topics are proposed as keyphrases. 1 Introduction Keyphrases are words, or groups of words, that capture the key ideas of a document. They repre- sent important information concerning a document and constitute an alternative, or a complement, to full-text indexing. Pertinent keyphrases are also useful to potential readers who can have a quick overview of the content of a document and can se- lect easily which document to read. Currently, the most powerful keyphrases extrac- tion algorithms are based on supervised learning. These methods address the problem of associat- ing keyphrases to documents as a classification task. However, the fact that this approach requires a corpus of similar documents, which is not al- ways readily available, constitutes a major draw- back. For example, if one encounters a new Web page, one might like to know quickly the main top- ics addressed. In this case, a domain-independent keyword extraction system that applies to a single document is needed. Several methods have been proposed for ex- tracting keywords from a single document (Mat- suo and Ishizuka, 2004; Palshikar, 2007). The re- ported performances were slightly higher than that obtained using a corpus and selecting the words with the highest TF-IDF 1 measure (Salton et al., 1975). The paper describes a new keyphrase extraction algorithm from a single document. We show that our system performs well without the need for a corpus. The paper is organized as follows. The next sec- tion describes the principles of our keyphrase ex- traction system. We present the main parts of the algorithm in section 3, we detail the methods in section 4 and we conclude the paper. 2 Principles When authors write documents, they have to think first at the way they will present their ideas. Most of the time, they establish content summaries that highlight the main topics of their texts. Then, they write the content of the documents by carefully selecting the most appropriate words to describe each topic. In this paper, we make the assumption that the words, or the set of words, that are repre- sentative of each topic constitute the keyphrases of the document. In the following of this paper, we call terms, the components of a …

Content truncated to 500 words