Unsupervised Bilingual Morpheme Segmentation and Alignment with Context-rich Hidden Semi-Markov Models

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 895–904, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Unsupervised Bilingual Morpheme Segmentation and Alignment with Context-rich Hidden Semi-Markov Models Jason Naradowsky∗ Department of Computer Science University of Massachusetts Amherst Amherst, MA 01003 narad@cs.umass.edu Kristina Toutanova Microsoft Research Redmond, WA 98502 kristout@microsoft.com Abstract This paper describes an unsupervised dynamic graphical model for morphological segmen- tation and bilingual morpheme alignment for statistical machine translation. The model ex- tends Hidden Semi-Markov chain models by using factored output nodes and special struc- tures for its conditional probability distribu- tions. It relies on morpho-syntactic and lex- ical source-side information (part-of-speech, morphological segmentation) while learning a morpheme segmentation over the target lan- guage. Our model outperforms a competi- tive word alignment system in alignment qual- ity. Used in a monolingual morphological seg- mentation setting it substantially improves ac- curacy over previous state-of-the-art models on three Arabic and Hebrew datasets. 1 Introduction An enduring problem in statistical machine trans- lation is sparsity. The word alignment models of modern MT systems attempt to capture p(ei|fj), the probability that token ei is a translation of fj. Underlying these models is the assumption that the word-based tokenization of each sentence is, if not optimal, at least appropriate for specifying a concep- tual mapping between the two languages. However, when translating between unrelated lan- guages – a common task – disparate morphological systems can place an asymmetric conceptual bur- den on words, making the lexicon of one language much more coarse. This intensifies the problem of sparsity as the large number of word forms created ∗This research was conducted during the author’s internship at Microsoft Research through morphologically productive processes hin- ders attempts to find concise mappings between con- cepts. For instance, Bulgarian adjectives may contain markings for gender, number, and definiteness. The following tree illustrates nine realized forms of the Bulgarian word for red, with each leaf listing the definite and indefinite markings. Feminine Neuter Singular Plural Root Masculine cherven(iq)(iqt) cherveni(te) chervena(ta) cherveno(to) Table 1: Bulgarian forms of red Contrast this with English, in which this informa- tion is marked either on the modified word or by sep- arate function words. In comparison to a language which isn’t mor- phologically productive on adjectives, the alignment model must observe nine times as much data (as- suming uniform distribution of the inflected forms) to yield a comparable statistic. In an area of research where the amount of data available plays a large role in a system’s overall performance, this sparsity can be extremely problematic. Further complications are created when lexical sparsity is compounded with the desire to build up alignments over increasingly larger contiguous phrases. To address this issue we propose an alternative to word alignment: morpheme alignment, an align- ment that operates over the smallest meaningful sub- sequences of words. By striving to keep a direct 1- to-1 mapping between corresponding semantic units across languages, we hope to find better estimates 895 ون the red …

Content truncated to 500 words