Usefulness of MT output for comprehension — an analysis from the point of view of linguistic intercomprehension

Usefulness of MT output for comprehension — an analysis from the point of view of linguistic intercomprehension Kenneth Jordan N´u˜nez kjordan@usj.es Universidad San Jorge, E-50830 Villanueva del G´allego, Spain Mikel L. Forcada mlf@ua.es Universitat d’Alacant, E-03690 Sant Vicent del Raspeig, Spain Esteve Clua esteve.clua@upf.edu Universitat Pompeu Fabra, E-08018 Barcelona, Spain Abstract The present paper describes and presents ongoing research on machine translation (MT) and linguistic intercomprehension. One main goal, although not the only one, is to evaluate three machine translation (MT) systems —Systran, Google Translate and Apertium— through an analysis of the readers’ ability to understand the output generated. We compare the usefulness of MT output for comprehension to that of non-native writing in the readers’ L1 and that of native writing in languages similar to their L1. The methodology used is based on cloze tests and the experiments are carried out using English, French and Italian as source languages and Spanish as the target language. The subjects involved are native Spanish first-year-undergraduate students and final-year secondary-school students. All of them have only very elementary knowledge, or in some cases no knowledge at all, of English, French and Italian (that is, a level equal to or lower than CEFRL B11). Although the results suggest that MT output resulting from translating from English and French into Spanish is similar to natively-written Italian texts or texts written in Spanish by non-native speakers in terms of usefulness, that depends quite often on the level of specialty but also on the field and on the MT system used. 1 Introduction The aim of the study, which is part of a broader research plan relating machine translation (MT) and linguistic intercomprehehsion, is to assess and compare three MT systems when used for assimilation2 or gisting – Systran, a hybrid system3 that combines statistical or corpus-based MT and rule-based MT; Google Translate,4 a statistical corpus-based MT system at the time of testing for the language pairs tested,5 and the Apertium rule-based system.6 The aim is achieved 1Common European Framework of Reference for Languages: Learning, Teaching, Assessment (http://www. coe.int/t/dg4/linguistic/Source/Framework\_EN.pdf) 2MT systems can be divided into two groups: those aimed at assimilation or gisting, which allow the user to understand the content of the text, and those aimed at dissemination, which helps to translate a text to be published. 3http://www.systran.es 4http://translate.google.com 5As of May 15, 2017, the English–Spanish system and the English–Italian systems are no longer statistical, but rather neural; the French–Spanish system is still statistical. 6http://www.apertium.org Proceedings of MT Summit XVI, vol.1: Research Track Nagoya, Sep. 18-22, 2017 | p. 241 by comparing the usefulness of their output to that of non-native writing in the reader’s first language L1 and to that of native writings (or MT output) in a closely related language. This will enable us to determine which of the three MT systems is the most useful for gisting, that is, which MT system results in the highest level of usefulness when reading a text originally written in a language unknown to the reader. Linguistic …

Content truncated to 500 words