Are ambiguous conjunctions problematic for machine translation?

Proceedings of Recent Advances in Natural Language Processing, pages 959–966, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_111 959 Are Ambiguous Conjunctions Problematic for Machine Translation? Maja Popovi´c and Sheila Castilho ADAPT Centre, School of Computing, DCU Ireland {firstname.lastname}@adaptcentre.ie Abstract The translation of ambiguous words still poses challenges for machine translation. In this work, we carry out a systematic quantitative analysis regarding the ability of different machine translation systems to disambiguate the source language con- junctions “but” and “and”. We evaluate specialised test sets focused on the trans- lation of these two conjunctions. The test sets contain source languages that do not distinguish different variants of the given conjunction, whereas the target languages do. In total, we evaluate the conjunc- tion “but” on 20 translation outputs, and the conjunction “and” on 10. All ma- chine translation systems almost perfectly recognise one variant of the target con- junction, especially for the source con- junction “but”. The other target variant, however, represents a challenge for ma- chine translation systems, with accuracy varying from 50% to 95% for “but” and from 20% to 57% for “and”. The major error for all systems is replacing the cor- rect target variant with the opposite one. 1 Introduction Ambiguous words are often difficult to translate automatically, even by the state-of-the-art neural machine (NMT) systems. Whereas the NMT ap- proach significantly improved fluency (grammar) of MT outputs compared to the previous state- of-the-art statistical phrase-based (PBMT) mod- els, adequacy (meaning preservation) is still often problematic (Castilho et al., 2017; Klubiˇcka et al., 2018). Adequacy is even more problematic for ambiguous words which have two or more mean- ings depending on the context. Therefore, ambiguity of nouns, verbs and pro- nouns has been investigated extensively in recent years (Guillou et al., 2018; M¨uller et al., 2018; Rios Gonzales et al., 2017, 2018). However, to the best of our knowledge, no results for ambi- guity of conjunctions have been reported so far. The only work dealing with conjunctions and ma- chine translation (Huang, 1983) explores conjunc- tion scope for rule-based MT systems and does not address the ambiguity. It should be noted, though, that the conjunction ambiguity is more structural than lexical: it is mainly related to certain aspects of grammar involving the arrangement of words and word types. Therefore, the conjunction ambi- guity is related more to fluency than it is to ade- quacy. In this work, we present the results of quanti- tative analysis addressing machine translation of two potentially ambiguous conjunctions, “but” and “and”. The analysis of the conjunction “but” is carried out for {English,French}-into- {Spanish,German,Serbian,Croatian} translation directions, and the conjunction “and” is analysed on {English,Portuguese}-into-{Serbian,Croatian} outputs. Evaluation is carried out on specialised test sets1 designed for evaluating translation of these ambiguous conjunctions. Instead of comparing the translation output with a reference human translation, our evaluation is based on presence or absence of the correct target language conjunction variant in the translation output. For a small number of sentences with both or with none of the target variants …

Content truncated to 500 words