Task-based Evaluation of Multiword Expressions: a Pilot Study in Statistical Machine Translation

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 242–245, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics Task-based Evaluation of Multiword Expressions: a Pilot Study in Statistical Machine Translation Marine Carpuat Mona Diab Columbia University Center for Computational Learning Systems 475 Riverside Drive, New York, NY 10115 {marine,mdiab}@ccls.columbia.edu Abstract We conduct a pilot study for task-oriented evaluation of Multiword Expression (MWE) in Statistical Machine Translation (SMT). We propose two different integration strategies for MWE in SMT, which take advantage of differ- ent degrees of MWE semantic compositional- ity and yield complementary improvements in SMT quality on a large-scale translation task.1 1 Introduction A multiword expression (MWE) generally refers to a multiword unit or a collocation of words that co- occur together statistically more than chance. MWE is a cover term for different types of colloca-A tions which vary in their transparency and fixedness. Identifying MWEs and understanding their meaning is considered essential to language understanding, and of crucial importance for any Natural Language Processing (NLP) applications that aim at handling robust language meaning and use. In fact, the sem- inal paper (Sag et al., 2002) refers to this problem as a key issue for the development of high-quality NLP applications. (Villavicencio et al., 2005) iden- tify Machine Translation as an application of partic- ular interest since “ recognition of MWEs is neces- sary for systems to preserve the meaning and pro- duce appropriate translations and avoid the genera- tion of unnatural or nonsensical sentences in the tar- get language.” However, statistical machine translation (SMT) typically does not model MWEs explicitly. SMT 1The research was partially funded by IBM under the DARPA GALE project. units are typically phrasal translations, defined with- out any direct syntactic or lexical semantic motiva- tion: they are simply n-grams that are consistently translated in parallel corpora. Phrasal translations might indirectly capture MWEs, but they are not dis- tinguished from any other n-gram. As a result, the usefulness of explicitly modeling MWEs in the SMT framework has not yet been stud- ied systematically. Previous work has focused on automatically learning and integrating translations of very specific MWE categories, such as, for in- stance, idiomatic Chinese four character expressions (Bai et al., 2009) or domain specific MWEs (Ren et al., 2009). MWEs have also been defined not from a lexical semantics perspective but from a SMT er- ror reduction perspective, as phrases that are hard to align during SMT training (Lambert and Banchs, 2005). For each of these particular cases, translation quality improved by augmenting the SMT transla- tion lexicon with the learned bilingual MWEs either directly or through improved word alignments. In this paper, we consider a more general prob- lem: we view SMT as an extrinsic evaluation of the usefulness of monolingual MWEs as used per- vasively in natural language regardless of domain, idiomaticity and compositionality. A MWE is com- positional if its meaning as a unit can be predicted from the meaning of its component …

Content truncated to 500 words