Word Sense Disambiguation Corpora Acquisition via Confirmation Code

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 1472–1476, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Word Sense Disambiguation Corpora Acquisition via Confirmation Code Wanxiang Che and Ting Liu∗ Research Center for Social Computing and Information Retrieval MOE-Microsoft Key Laboratory of Natural Language Processing and Speech School of Computer Science and Technology Harbin Institute of Technology, China {car, tliu}@ir.hit.edu.cn Abstract Word Sense Disambiguation (WSD) is one of the fundamental natural language pro- cessing tasks. However, lack of train- ing corpora is a bottleneck to construct a high accurate all-words WSD system. Annotating a large-scale corpus by ex- perts costs enormous time and financial re- sources. Human Computation is a novel idea for integrating human resources be- hind the Web, which has been wasted, to solve practical problems that are dif- ficult for computers. Based on human computation, we design a confirmation code system, which can not only distin- guish between human beings and comput- ers (the function of normal confirmation code system), but also annotate WSD cor- pora. The preliminary experimental result shows that the proposed method can an- notate large-scale and high-quality WSD corpora within a short time. To the best of our knowledge, this is the first attempt to use confirmation code in natural language processing for corpora acquisition. 1 Introduction It is a common phenomenon that a word has mul- tiple senses in natural languages. The aim of word sense disambiguation (WSD) is to identify the cor- rect senses of ambiguous words according to their surrounding contexts. WSD is a basic task of nat- ural language processing. The state-of-the-art in WSD is dominated by supervised machine learn- ing methods where a model is trained to recognize word senses in a given context based on various features. Although it is important to use pow- erful machine learning algorithms, latest studies ∗Correspondence author: tliu@ir.hit.edu.cn have found that large-scale and high-quality cor- pora are more important for WSD (Agirre and Ed- monds, 2006). Therefore, building such corpora is key challenge to be addressed. Currently, corpora are created mainly through manual annotation by expert annotators. How- ever, the cost of annotating a necessary size of cor- pora is prohibitive. Consequently, in the research field of WSD, some common ambiguous words are sampled and then annotated with a necessary amount of examples. The sampling method pro- motes the research in WSD algorithms. However, these algorithms are difficult to be used in practi- cal applications due to lack of large-scale corpora in which all ambiguous words are annotated. For example, SemCor1 corpus contains WSD annota- tion of about 250,000 words sampled from a sub- set of the Brown corpus. However, for most of the ambiguous words, the number of examples is still too small to train a high performance all-words WSD model. The best performance of Senseval- 3 English all-words evaluation task (Snyder and Palmer, 2004) is only about 65%. Semi-supervised methods have been applied to build large-scale corpora, such as bootstrap- ping (Yarowsky, 1995). However, …

Content truncated to 500 words