Czech Text Document Corpus v 2.0

Czech Text Document Corpus v 2.0 Pavel Kr´al1,2, Ladislav Lenc1,2 1Dept. of Computer Science & Engineering 2New Technologies for the Information Society Faculty of Applied Sciences Faculty of Applied Sciences University of West Bohemia University of West Bohemia Plzeˇn, Czech Republic Plzeˇn, Czech Republic Abstract This paper introduces “Czech Text Document Corpus v 2.0”, a collection of text documents for automatic document classification in Czech language. It is composed of the text documents provided by the Czech News Agency and is freely available for research purposes at http://ctdc.kiv.zcu.cz/. This corpus was created in order to facilitate a straightforward comparison of the document classification approaches on Czech data. It is particularly dedicated to evaluation of multi-label document classification approaches, because one document is usually labelled with more than one label. Besides the information about the document classes, the corpus is also annotated at the morphological layer. This paper further shows the results of selected state-of-the-art methods on this corpus to offer the possibility of an easy comparison with these approaches. Keywords: corpus, Czech, document classification, multi-label, text 1. Introduction Automatic classification (or categorization) of text docu- ments is very important for information organization and storage because of the significant increase of the amount of electronic documents and the rapid growth of the Internet. Many efficient approaches have been proposed. They are usually based on supervised machine learning. The docu- ments are projected into the so-called vector space model, basically using the words as features for various classifica- tion algorithms. The approaches differ in the used methods, however the common point is that all of them need an an- notated document corpus to train the parameters. A sufficient number of the corpora in several languages, particularly in English, is freely available. However, to the best of our knowledge, the Czech one is missing. The main goal of this paper consists in presenting a corpus of Czech text documents. It is composed of real newspaper articles provided by the Czech News Agency ( ˇCTK)1 and is available for research purposes for free. It is created for a straightforward comparison of the document classifica- tion approaches in Czech. One document is usually labelled with more than one label, therefore this corpus is usually used for evaluation of multi-label document classification. Besides the information about the document classes, the corpus is also morphologically annotated. The morpholog- ical annotation has been done fully automatically. Another research contribution of this paper represents the reported results of selected state-of-the-art methods on this corpus to offer the possibility of an easy comparison with these approaches. The paper structure is as follows. The following sec- tion presents other text corpora for document classification freely available for research purposes. Section 3. details our corpus. Section 4. presents the results of the selected state-of-the art methods on this dataset. The last section concludes the paper. 1http://www.ctk.eu/ 2. Other Text Corpora Some important existing text classification corpora in sev- eral languages are described below. 2.1. Reuters-21578 Reuters-215782 corpus is a collection of 21,578 …

Content truncated to 500 words