Building Domain-Specific Taggers without Annotated (Domain) Data

Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 1103–1111, Prague, June 2007. c⃝2007 Association for Computational Linguistics Building Domain-Specific Taggers without Annotated (Domain) Data John E. Miller1 Manabu Torii2 K. Vijay-Shanker1 1Computer & Information Sciences University of Delaware Newark, DE 19716 {jmiller,vijay}@cis.udel.edu 2Biostatistics, Bioinformatics and Biomathematics Georgetown University Medical Center Washington, DC 20057 mt352@georgetown.edu Abstract Part of speech tagging is a fundamental component in many NLP systems. When taggers developed in one domain are used in another domain, the performance can degrade considerably. We present a method for developing taggers for new domains without requiring POS annotated text in the new domain. Our method involves using raw domain text and identifying related words to form a domain specific lexicon. This lexicon provides the initial lexical probabilities for EM training of an HMM model. We evaluate the method by apply- ing it in the Biology domain and show that we achieve results that are comparable with some taggers developed for this domain. 1 Introduction As Natural Language Processing (NLP) technol- ogy advances and more text becomes available, it is being applied more and often in specialized do- mains. Part of Speech (POS) tagging is often a fundamental component to these NLP applications and hence its accuracy can have a significant im- pact on the application’s success. The success that the taggers have attained is often not replicated when the domain is changed. Degradation of accu- racy in a new domain can be overcome by devel- oping an annotated corpus for that specific domain, e.g., as in the Biology domain. However, this solu- tion is feasible only if there is sufficient interest in the use of NLP technology in that domain, and there are sufficient funding and resources. In con- trast, our approach is to use existing resources, and rapidly develop taggers for new domains without using the time and effort to develop annotated data. In this work, we use the Wall Street Journal (WSJ) corpus (Marcus et al, 1993) and large amounts of domain-specific raw text to develop taggers. We evaluate our methodology in the Biol- ogy domain and show the resulting performance is competitive with some taggers built with super- vised learning for that domain. Also, we note that the accuracy of taggers trained on the WSJ corpus drops off considerably when applied to this domain. Smith et al. (2005) report that the Brill tagger (1995) has an accuracy of 86.8% on 1000 sen- tences taken from Medline, and that the Xerox tag- ger (Cutting et al .1992) has an accuracy of 93.1% on the same sentences. They attribute this drop off to the fact that only 57.8% of the 10,000 most fre- quent words can be found in WSJ corpus. This ob- servation provides further impetus to developing lexicon for taggers in the new domains. In the next section, we discuss our general ap- proach. The details of the EM training of the HMM tagger are given in …

Content truncated to 500 words