Machine Translation Detection from Monolingual Web-Text

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 1597–1607, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics Machine Translation Detection from Monolingual Web-Text Yuki Arase Microsoft Research Asia No. 5 Danling St., Haidian Dist. Beijing, P.R. China yukiar@microsoft.com Ming Zhou Microsoft Research Asia No. 5 Danling St., Haidian Dist. Beijing, P.R. China mingzhou@microsoft.com Abstract We propose a method for automatically detecting low-quality Web-text translated by statistical machine translation (SMT) systems. We focus on the phrase salad phenomenon that is observed in existing SMT results and propose a set of computa- tionally inexpensive features to effectively detect such machine-translated sentences from a large-scale Web-mined text. Un- like previous approaches that require bilin- gual data, our method uses only monolin- gual text as input; therefore it is applicable for refining data produced by a variety of Web-mining activities. Evaluation results show that the proposed method achieves an accuracy of 95.8% for sentences and 80.6% for text in noisy Web pages. 1 Introduction The Web provides an extremely large volume of textual content on diverse topics and areas. Such data is beneficial for constructing a large scale monolingual (Microsoft Web N-gram Ser- vices, 2010; Google N-gram Corpus, 2006) and bilingual (Nie et al., 1999; Shi et al., 2006; Ishisaka et al., 2009; Jiang et al., 2009) corpus that can be used for training statistical models for NLP tools, as well as for building a large-scale knowledge-base (Suchanek et al., 2007; Zhu et al., 2009; Fader et al., 2011; Nakashole et al., 2012). With recent advances in statistical machine trans- lation (SMT) systems and their wide adoption in Web services through APIs (Microsoft Translator, 2009; Google Translate, 2006), a large amount of text in Web pages is translated by SMT sys- tems. According to Rarrick et al. (2011), their Web crawler finds that more than 15% of English- Japanese parallel documents are machine transla- tion. Machine-translated sentences are useful if they are of sufficient quality and indistinguish- able from human-generated sentences; however, the quality of these machine-translated sentences is generally much lower than sentences generated by native speakers and professional translators. Therefore, a method to detect and filter such SMT results is desired to best make use of Web-mined data. To solve this problem, we propose a method for automatically detecting Web-text translated by SMT systems1. We especially target machine- translated text produced through the Web APIs that is rapidly increasing. We focus on the phrase salad phenomenon (Lopez, 2008), which char- acterizes translations by existing SMT systems, i.e., each phrase in a sentence is semantically and syntactically correct but becomes incorrect when combined with other phrases in the sentence. Based on this trait, we propose features for eval- uating the likelihood of machine-translated sen- tences and use a classifier to determine whether the sentence is generated by the SMT systems. The primary contributions of the proposed method are threefold. First, unlike previous stud- ies that use parallel text and bilingual features, such as (Rarrick et al., …

Content truncated to 500 words