Unsupervised Translation Induction for Chinese Abbreviations using Monolingual Corpora

Proceedings of ACL-08: HLT, pages 425–433, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics Unsupervised Translation Induction for Chinese Abbreviations using Monolingual Corpora Zhifei Li and David Yarowsky Department of Computer Science and Center for Language and Speech Processing Johns Hopkins University, Baltimore, MD 21218, USA zhifei.work@gmail.com and yarowsky@cs.jhu.edu Abstract Chinese abbreviations are widely used in modern Chinese texts. Compared with English abbreviations (which are mostly acronyms and truncations), the formation of Chinese abbreviations is much more complex. Due to the richness of Chinese abbreviations, many of them may not appear in available par- allel corpora, in which case current machine translation systems simply treat them as un- known words and leave them untranslated. In this paper, we present a novel unsupervised method that automatically extracts the relation between a full-form phrase and its abbrevia- tion from monolingual corpora, and induces translation entries for the abbreviation by us- ing its full-form as a bridge. Our method does not require any additional annotated data other than the data that a regular translation system uses. We integrate our method into a state-of- the-art baseline translation system and show that it consistently improves the performance of the baseline system on various NIST MT test sets. 1 Introduction The modern Chinese language is a highly abbrevi- ated one due to the mixed use of ancient single- character words with modern multi-character words and compound words. According to Chang and Lai (2004), approximately 20% of sentences in a typical news article have abbreviated words in them. Ab- breviations have become even more popular along with the development of Internet media (e.g., online chat, weblog, newsgroup, and so on). While En- glish words are normally abbreviated by either their Full-form Abbreviation Translation �� � � �� � � �� Hong Kong Governor � � �� � � ��� � � ��� Security Council Figure 1: Chinese Abbreviations Examples first letters (i.e. acronyms) or via truncation, the for- mation of Chinese abbreviations is much more com- plex. Figure 1 shows two examples for Chinese ab- breviations. Clearly, an abbreviated form of a word can be obtained by selecting one or more characters from this word, and the selected characters can be at any position in the word. In an extreme case, there are even re-ordering between a full-form phrase and its abbreviation. While the research in statistical machine trans- lation (SMT) has made significant progress, most SMT systems (Koehn et al., 2003; Chiang, 2007; Galley et al., 2006) rely on parallel corpora to extract translation entries. The richness and complexness of Chinese abbreviations imposes challenges to the SMT systems. In particular, many Chinese abbrevi- ations may not appear in available parallel corpora, in which case current SMT systems treat them as unknown words and leave them untranslated. This affects the translation quality significantly. To be able to translate a Chinese abbreviation that is unseen in available parallel corpora, one may an- notate more parallel data. However, this is very expensive as there are too many …

Content truncated to 500 words