Teaching machine translation evaluation by assessed project work

131 Teaching Machine Translation evaluation by assessed project work Judith Belam Foreign Language Centre, University of Exeter Queen’s Building The Queen’s Drive Exeter EX4 4QH UK J.M.Belam@ex.ac.uk Abstract The paper describes the use of an assessed independent study project on machine translation evaluation as part of a final-year undergraduate course on machine assisted translation. The advantages and potential drawbacks of the use of such a component are outlined. The usefulness of studying MT evaluation is underlined, as it includes consideration of many aspects of MT. The suitability of teaching MT to language learners is considered, as are the transferable skills students can hope to gain after following the course. Introduction The course, entitled “Machine-Assisted Translation” (MAT) is for final-year undergraduates in Modern Languages. It runs over one semester and is worth 15 credits of a 120-credit study programme. Students will have achieved a good working competence in at least one foreign language and will have spent a year in the target language country. They may have used computer-assisted language learning materials but no knowledge of MAT tools is assumed before they start the course. The MAT course is not primarily intended as a language teaching course. Students will continue their language learning concurrently on a course consisting of oral classes, essay writing and short literary texts for translation into and out of the target language. The MAT course on the other hand aims to provide a broadly based introduction to MAT including the basics of text processing, practical aspects like pre- and post-editing and dictionary creation, and the history of MAT. The assessment of the course falls into two parts. 50% of the marks are given for performance in a 2-hour written examination and 50% for an independent study project. The independent study project consists of an evaluation of one or more MT systems. The guidelines given to the students are as follows: The portfolio takes the form of a report on an evaluation of a machine translation system, which you will design and carry out. You will be studying the subject of machine translation evaluation formally in weeks 6 and 7, and you will then be ready to proceed with the preparation of your portfolio. It is expected that you will begin with a short introduction outlining the principles of machine translation evaluation; this will include types of evaluation and questions of why, by whom and for whom evaluations may be carried out. You will then describe the type of evaluation you yourself have decided to do: you may be focussing on output (accuracy, readability, coherence …) or on the system itself (ease of use of dictionaries, number of additional features available, handling of text within word processing programs …); or you may have decided to look at the way the program handles specific problems (subject domains, or specific grammatical structures); or you may address a practical problem like how much texts need to be pre- or post-edited in order to arrive at an acceptable translation. You will give …

Content truncated to 500 words