Integrating Joint n-gram Features into a Discriminative Training Framework

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 697–700, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics Integrating Joint n-gram Features into a Discriminative Training Framework Sittichai Jiampojamarn† and Colin Cherry‡ and Grzegorz Kondrak† †Department of Computing Science ‡National Research Council Canada University of Alberta 1200 Montreal Road Edmonton, AB, T6G 2E8, Canada Ottawa, ON, K1A 0R6, Canada {sj,kondrak}@cs.ualberta.ca Colin.Cherry@nrc-cnrc.gc.ca Abstract Phonetic string transduction problems, such as letter-to-phoneme conversion and name transliteration, have recently received much attention in the NLP community. In the past few years, two methods have come to dom- inate as solutions to supervised string trans- duction: generative joint n-gram models, and discriminative sequence models. Both ap- proaches benefit from their ability to consider large, flexible spans of source context when making transduction decisions. However, they encode this context in different ways, provid- ing their respective models with different in- formation. To combine the strengths of these two systems, we include joint n-gram fea- tures inside a state-of-the-art discriminative sequence model. We evaluate our approach on several letter-to-phoneme and translitera- tion data sets. Our results indicate an improve- ment in overall performance with respect to both the joint n-gram approach and traditional feature sets for discriminative models. 1 Introduction Phonetic string transduction transforms a source string into a target representation according to its pronunciation. Two important examples of this task are letter-to-phoneme conversion and name translit- eration. In general, the problem is challenging be- cause source orthography does not unambiguously specify the target representation. When consider- ing letter-to-phoneme, ambiguities and exceptions in the pronunciation of orthography complicate con- version. Transliteration suffers from the same ambi- guities, but the transformation is further complicated by restrictions in the target orthography that may not exist in the source. Joint n-gram models (Bisani and Ney, 2002; Chen, 2003; Bisani and Ney, 2008) have been widely applied to string transduction problems (Li et al., 2004; Demberg et al., 2007; Jansche and Sproat, 2009). The power of the approach lies in building a language model over the operations used in the conversion from source to target. Crucially, this al- lows the inclusion of source context in the generative story. Smoothing techniques play an important role in joint n-gram models, greatly affecting their per- formance. Although joint n-gram models are capa- ble of capturing context information in both source and target, they cannot selectively use only source or target information, nor can they consider arbitrary sequences within their context window, as they are limited by their back-off schedule. Discriminative sequence models have also been shown to perform extremely well on string transduc- tion problems. These begin with a Hidden Markov Model architecture, augmented with substring op- erations and discriminative training. The primary strength of these systems is their ability to include rich indicator features representing long sequences of source context. We will assume a specific in- stance of discriminative sequence modeling, DI- RECTL (Jiampojamarn et al., 2009), which achieved the best results on several …

Content truncated to 500 words