Unsupervised Part-of-Speech Acquisition for Resource-Scarce Languages

Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 218–227, Prague, June 2007. c⃝2007 Association for Computational Linguistics Unsupervised Part-of-Speech Acquisition for Resource-Scarce Languages Sajib Dasgupta and Vincent Ng Human Language Technology Research Institute University of Texas at Dallas Richardson, TX 75083-0688 {sajib,vince}@hlt.utdallas.edu Abstract This paper proposes a new bootstrapping approach to unsupervised part-of-speech induction. In comparison to previous bootstrapping algorithms developed for this problem, our approach aims to improve the quality of the seed clusters by employing seed words that are both distributionally and morphologically reliable. In particular, we present a novel method for combining morphological and distributional information for seed selection. Experimental results demonstrate that our approach works well for English and Bengali, thus providing suggestive evidence that it is applicable to both morphologically impoverished languages and highly inflectional languages. 1 Introduction The availability of a high-quality lexicon is crucial to the development of fundamental text-processing components such as part-of-speech (POS) taggers and syntactic parsers. While hand-crafted lexicons are readily available for resource-rich languages such as English, the same is not true for resource- scarce languages. Unfortunately, manually constructing a lexicon requires a lot of linguistic expertise, and is practically infeasible for highly inflectional and agglutinative languages, which contain a very large number of lexical items. Given the scarcity of annotated data for acquiring the lexicon in a supervised manner, researchers have instead investigated unsupervised POS induction techniques for automating the lexicon construction process. In essence, the goal of unsupervised POS induction is to learn the set of possible POS tags for each lexical item from an unannotated corpus. The most common approach to unsupervised POS induction to date has been motivated by Har- ris’s (1954) distributional hypothesis: words with similar co-occurrence patterns should have similar syntactic behavior. More specifically, unsupervised POS induction algorithms typically operate by (1) representing each target word (i.e., a word to be tagged with its POS) as a context vector that en- codes its left and right context, (2) clustering dis- tributionally similar words, and (3) manually label- ing each cluster with a POS tag by inspecting the members of the cluster. This distributional approach works under the as- sumption that the context vector of each word en- codes sufficient information for enabling accurate word clustering. However, many words are dis- tributionally unreliable: due to data sparseness, they occur infrequently and hence their context vectors do not capture reliable statistical informa- tion. To overcome this problem, Clark (2000) pro- poses a bootstrapping approach, in which he (1) clusters the most distributionally reliable words, and then (2) incrementally augments each cluster with words that are distributionally similar to those already in the cluster. The goal of this paper is to propose a new boot- strapping approach to unsupervised POS induction that can operate in a resource-scarce setting. Most notably, our approach aims to improve the quality of the seed clusters by employing seed words that are both distributionally and morphologically reli- able. In particular, …

Content truncated to 500 words