Training Conditional Random Fields Using Incomplete Annotations

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 897–904 Manchester, August 2008 Training Conditional Random Fields Using Incomplete Annotations Yuta Tsuboi, Hisashi Kashima Tokyo Research Laboratory, IBM Research, IBM Japan, Ltd Yamato, Kanagawa 242-8502, Japan {yutat,hkashima}@jp.ibm.com Shinsuke Mori Academic Center for Computing and Media Studies, Kyoto University Sakyo-ku, Kyoto 606-8501, Japan forest@i.kyoto-u.ac.jp Hiroki Oda Shinagawa, Tokyo, Japan oda@fw.ipsj.or.jp Yuji Matsumoto Graduate School of Information Science, Nara Institute of Science and Technology Takayama, Ikoma, Nara 630-0101, Japan matsu@is.naist.jp Abstract We address corpus building situations, where complete annotations to the whole corpus is time consuming and unrealistic. Thus, annotation is done only on crucial part of sentences, or contains unresolved label ambiguities. We propose a parame- ter estimation method for Conditional Ran- dom Fields (CRFs), which enables us to use such incomplete annotations. We show promising results of our method as applied to two types of NLP tasks: a domain adap- tation task of a Japanese word segmenta- tion using partial annotations, and a part- of-speech tagging task using ambiguous tags in the Penn treebank corpus. 1 Introduction Annotated linguistic corpora are essential for building statistical NLP systems. Most of the corpora that are well-known in NLP communi- ties are completely-annotated in general. However it is quite common that the available annotations are partial or ambiguous in practical applications. For example, in domain adaptation situations, it is time-consuming to annotate all of the elements in a sentence. Rather, it is efficient to annotate certain parts of sentences which include domain-specific expressions. In Section 2.1, as an example of such efficient annotation, we will describe the effective- ness of partial annotations in the domain adapta- tion task for Japanese word segmentation (JWS). In addition, if the annotators are domain experts c⃝ 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported li- cense (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. rather than linguists, they are unlikely to be confi- dent about the annotation policies and may prefer to be allowed to defer some linguistically complex decisions. For many NLP tasks, it is sometimes difficult to decide which label is appropriate in a particular context. In Section 2.2, we show that such ambiguous annotations exist even in a widely used corpus, the Penn treebank (PTB) corpus. This motivated us to seek to incorporate such incomplete annotations into a state of the art ma- chine learning technique. One of the recent ad- vances in statistical NLP is Conditional Random Fields (CRFs) (Lafferty et al., 2001) that evaluate the global consistency of the complete structures for both parameter estimation and structure infer- ence, instead of optimizing the local configurations independently. This feature is suited to many NLP tasks that include correlations between elements in the output structure, such as the interrelation of part-of-speech (POS) tags in a sentence. However, conventional CRF algorithms require fully anno- tated sentences. To incorporate incomplete anno- tations into CRFs, we extend the structured out- put problem in Section 3. We focus on partial an- notations or ambiguous annotations in this …

Content truncated to 500 words