Labeling the Languages of Words in Mixed-Language Documents using Weakly Supervised Methods

Proceedings of NAACL-HLT 2013, pages 1110–1119, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Labeling the Languages of Words in Mixed-Language Documents using Weakly Supervised Methods Ben King Department of EECS University of Michigan Ann Arbor, MI benking@umich.edu Steven Abney Department of Linguistics University of Michigan Ann Arbor, MI abney@umich.edu Abstract In this paper we consider the problem of label- ing the languages of words in mixed-language documents. This problem is approached in a weakly supervised fashion, as a sequence la- beling problem with monolingual text sam- ples for training data. Among the approaches evaluated, a conditional random field model trained with generalized expectation criteria was the most accurate and performed consis- tently as the amount of training data was var- ied. 1 Introduction Language identification is a well-studied problem (Hughes et al., 2006), but it is typically only studied in its canonical text-classification formulation, iden- tifying a document’s language given sample texts from a few different languages. But there are sev- eral other interesting and useful formulations of the problem that have received relatively little attention. Here, we focus on the problem of labeling the lan- guages of individual words within a multilingual document. To our knowledge, this is the first paper to specifically address this problem. Our own motivation for studying this problem stems from issues encountered while attempting to build language resources for minority languages. In trying to extend parts of Kevin Scannell’s Cr´ubad´an project (Scannell, 2007), which automatically builds minority language corpora from the Web, we found that the majority of webpages that contain text in a minority language also contain text in other lan- guages. Since Scannell’s method builds these cor- pora by bootstrapping from the pages that were re- trieved, the corpus-building process can go disas- trously wrong without accounting for this problem. And any resources, such as a lexicon, created from the corpus will also be incorrect. In this paper, we explore techniques for per- forming language identification at the word level in mixed language documents. Our results show that one can do better than independent word language classification, as there are clues in a word’s context: words of one language are frequently surrounded by words in the same language, and many documents have patterns that may be marked by the presence of certain words or punctuation. The methods in this paper also outperform sentence-level language iden- tification, which is too coarse to capture most of the shifts between language. To evaluate our methods, we collected and man- ually annotated a corpus of over 250,000 words of bilingual (though mostly non-parallel) text from the web. After running several different weakly- supervised learning methods, we found that a condi- tional random field model trained with generalized expectation criteria is the most accurate and per- forms quite consistently as the amount of training data is varied. In section 2, we review the related work. In sec- tion 3, we define the task and describe the data and its annotation. Because the task of …

Content truncated to 500 words