Linguistically-Motivated Yorùbá-English Machine Translation

Proceedings of the 29th International Conference on Computational Linguistics, pages 5066–5075 October 12–17, 2022. 5066 Linguistically-Motivated Yorùbá-English Machine Translation Ife Adebara1,2 Muhammad Abdul Mageed1,2 Miikka Silfverberg1 Department of Linguistics1 Deep Learning and Natural Language Processing Group2 The University of British Columbia {ife.adebara, muhammad.mageed, miikka.silfverberg}@ubc.ca Abstract Translating between languages where certain features are marked morphologically in one but absent or marked contextually in the other is an important test case for machine translation. When translating into English which marks (in)definiteness morphologically, from Yorùbá which uses bare nouns but marks these features contextually, ambiguities arise. In this work, we perform fine-grained analysis on how an SMT system compares with two NMT systems (BiLSTM and Transformer) when translating bare nouns in Yorùbá into English. We investi- gate how and to what extent the systems iden- tify BNs, correctly translate them, and compare with human translation patterns. We also ana- lyze the type of errors each model makes and provide a linguistic description of these errors. We glean insights for evaluating model perfor- mance in low-resource settings. In translating bare nouns, our results show the transformer model outperfoms the SMT and BiLSTM mod- els for 4 categories, the BiLSTM outperforms the SMT model for 3 categories while the SMT outperforms the NMT models for 1 category. 1 Introduction Languages differ with regard to how grammati- cal information such as “case" and “number" are expressed. In some languages, this information is overtly marked using morphological or syntac- tic means, whereas in others it has to be inferred from context. This asymmetry of information repre- sentation poses an important problem for machine translation (Mitkov, 1999; Hardmeier, 2012). Sev- eral phenomena where asymmetry arises have been identified as challenging problems for machine translation. These include: pronoun translation and coreference (Guillou et al., 2019), politeness (Sennrich et al., 2016), lexical cohesion (Carpuat, 2009), and lexical disambiguation (Gonzales et al., 2017). Asymmetry of information representation presents an interesting test case for MT systems because it can shed light on their true linguistic ability (Voita et al., 2018; Bawden et al., 2017). It is, therefore, important to use evaluation measures which can capture this aspect of the translation task. However, the most popular evaluation metric for machine translation, the BLEU score (Papineni et al., 2002a), is a coarse metric which can often hide these fine-grained morphological and semantic distinctions. In fact, a high BLEU score is no guarantee of improved translation quality and BLEU, being based on precision on short ngrams, may be poorly suited for measuring the coherence and grammaticality of a sentence. In this paper we investigate the performance of Yorùbá 1 to English Machine Translation. We specifically evaluate performance on translating Bare nouns (BNs). BNs (Cheng and Sybesma, 1999; Krifka, 2003; Chierchia, 1998; Larson, 1985; Carlson, 1989) are nouns without an overt de- terminer or quantifier. For instance "houses" in Houses are expensive in New York is a BN. Whereas English accounts for only plural BNs, BNs in Yorùbá are number neutral and can also be definite …

Content truncated to 500 words