Penn Korean Treebank : Development and Evaluation

Penn Korean Treebank: Development and Evaluation Chung-hye Han Dept. of Linguistics Simon Fraser University 8888 University Drive Burnaby BC V5A 156, Canada chunghye@sfu.ca Martha Palmer Dept. of Computer Information and Science University of Pennsylvani 256 Moore School Philadephia, PA 19104, USA mpalmer@linc.cis.upenn.edu Na-Rae Han, Eon-Suk Ko Dept. of Linguistics University of Pennsylvania 619 Williams Hall Philadelphia, PA 19104, USA Inrh,eskolAling.upenn.edu Heejong Yi Dept. of Lingistics University of Delaware 46E. Delaware Ave. Newark, DE 19716, USA hyi©udel.edu Abstract This paper discusses issues in building a 54-thousand-word Korean Treebank using a phrase structure annotation, along with developing annotation guidelines based on the morpho-syntactic phenomena represented in the corpus. Various methods that were employed for quality control and the evaluation on the Treebank are also presented. 1 Introduction With growing interest in Korean language processing, numerous natural languages processing (NLP) tools for Korean, such as part-of-speech (POs) taggers, morphological analyzers , parsers, have been developed. This progress was possible through the availability of large-scale raw text corpora and POS tagged corpora (ETRI, 1999; Yoon and Choi, 1999a; Yoon and Choi, 1999b). However, no large-scale bracketed corpora are currently available to the public, although efforts have been made to develop guidelines for syntactic annotation (Lee et al., 1996; Lee et al., 1997). As a first step towards addressing this issue, we built a 54-thousand-word s Korean Treebank using a phrase structure annotation. At the same time, we also developed annotation guidelines based on the morpho-syntactic phenomena represented in the corpus, over the period of Jan. 2000 and April 2001. The corpus that we used for the Korean Treebank consists of texts from military language training manuals. These texts contain information about various aspects of the military, such as troop movement, intelligence gathering, and equipment supplies, among others. This corpus is part of a Korean/English bilingual corpora that was used for domain specific Korean/English machine translation project at the University of Pennsylvania. One of the main reasons for annotating this corpus was to train taggers and parsers that can be used for the MT project. In this paper, we first discuss some issues in developing the annotation guidelines for POS tagging and syntactic bracketing. We then detail the annotation process in §3, including various methods we used to detect and correct annotation errors. §4 presents some statistics on the size of the corpus, and §5 discusses the results of the evaluation on the Treebank. 2 Guideline development The guiding principles employed in developing the annotation guidelines were theory-neutralness (whenever possible), descriptive accuracy and consistency. To this end, various existing knowl- edge sources were consulted, including theoretical linguistic literature on Korean, publications 'This word count is computed on tokenized texts and includes symbols. 69 on Korean descriptive grammar, as well as research works on building tagged Korean copora by such institutions as KAIST and ETRI (ETRI, 1999; Lee et al., 1996; Lee et al., 1997; Yoon and Choi, 1999a, Yoon and Choi, 1999b). Ideally, complete guidelines should be available to the an- notators before annotation begins. …

Content truncated to 500 words