Part-of-Speech Tagging for English-Spanish Code-Switched Text

Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 1051–1060, Honolulu, October 2008. c⃝2008 Association for Computational Linguistics Part-of-Speech Tagging for English-Spanish Code-Switched Text Thamar Solorio and Yang Liu Human Language Technology Research Institute The University of Texas at Dallas Richardson, TX 75080, USA tsolorio,yangl@hlt.utdallas.edu Abstract Code-switching is an interesting linguistic phenomenon commonly observed in highly bilingual communities. It consists of mixing languages in the same conversational event. This paper presents results on Part-of-Speech tagging Spanish-English code-switched dis- course. We explore different approaches to exploit existing resources for both languages that range from simple heuristics, to language identification, to machine learning. The best results are achieved by training a machine learning algorithm with features that combine the output of an English and a Spanish Part- of-Speech tagger. 1 Introduction Worldwide the percentage of bilingual speakers is fairly large, and it keeps increasing at a high rate. In the U.S., 18% of the total population speaks a language other than English at home, the major- ity of which speaks Spanish (U.S. Census Bureau, 2003). A significant percentage of this Spanish- English bilingual population code-switch between the two languages in what is often referred as Span- glish, the mix of Spanish and English. Spanish and English are not the only occurrence of language mixtures. Examples of other popular combinations include Arabic dialects, French and German, Span- ish and Catalan, Maltese and English, and English and French. Typically when there are linguistic bor- ders, or when the country has more than one official language, we can find instances of code-switching. Despite the wide use of code-switched discourse among bilinguals, this linguistic phenomenon has received little attention in the fields of Natural Lan- guage Processing and Computational Linguistics. Part-of-Speech (POS) tagging is a well studied prob- lem in these fields. For languages such as English, German, Spanish, and Chinese there are several dif- ferent POS taggers that reach high accuracies, espe- cially in news text corpora. However, to our knowl- edge, there is no previous work on developing a POS tagger for text with mixes of languages. In this paper we present results on the problem of POS tagging English-Spanish code-switched dis- course by taking advantage of existing taggers for both languages. This rationale follows the evi- dence from studies of code-switching on different language pairs, which have shown code-switching to be grammatical according to both languages be- ing switched. We use different heuristics to combine POS tag information from existing monolingual tag- gers. We also explore the use of different language identification methods to select POS tags from the appropriate monolingual tagger. However, the best results are achieved by a machine learning approach using features generated by the monolingual POS taggers. The next section presents the facts about code- switching, including some previous work done mainly in linguistics. Then in Section 3 we dis- cuss previous work related to the automated pro- cessing of code-switched discourse. In Section 4 we describe the English-Spanish code-switched data set gathered for the …

Content truncated to 500 words