Findings of the First WMT Shared Task on Sign Language Translation (WMT-SLT22)

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 744 - 772 December 7-8, 2022 ©2022 Association for Computational Linguistics Findings of the First WMT Shared Task on Sign Language Translation (WMT-SLT22) Mathias Müller University of Zurich Sarah Ebling University of Zurich Eleftherios Avramidis DFKI Berlin Alessia Battisti University of Zurich Michèle Berger HfH Zurich Richard Bowden University of Surrey Annelies Braffort University of Paris-Saclay Necati Cihan Camgöz Meta Reality Labs Cristina España-Bonet DFKI Saarbrücken Roman Grundkiewicz Microsoft Zifan Jiang University of Zurich Oscar Koller Microsoft Amit Moryossef Bar-Ilan University Regula Perrollaz HfH Zurich Sabine Reinhard HfH Zurich Annette Rios University of Zurich Dimitar Shterionov Tilburg University Sandra Sidler-Miserez HfH Zurich Katja Tissi HfH Zurich Davy Van Landuyt European Union of the Deaf Abstract This paper presents the results of the First WMT Shared Task on Sign Language Trans- lation (WMT-SLT22)1. This shared task is concerned with automatic translation between signed and spoken2 languages. The task is novel in the sense that it requires process- ing visual information (such as video frames or human pose estimation) beyond the well- known paradigm of text-to-text machine trans- lation (MT). The task featured two tracks, translating from Swiss German Sign Language (DSGS) to German and vice versa. Seven teams participated in this first edition of the task, all submitting to the DSGS-to-German track. Besides a system ranking and sys- tem papers describing state-of-the-art tech- niques, this shared task makes the following scientific contributions: novel corpora, repro- ducible baseline systems and new protocols and software for human evaluation. Finally, the task also resulted in the first publicly avail- able set of system outputs and human evalua- tion scores for sign language translation. 1 Introduction This paper presents the outcome of the First WMT Shared Task on Sign Language Transla- tion (WMT-SLT22). The focus of this shared task is automatic translation between signed and 1https://www.wmt-slt.com/ 2In this paper we use the word “spoken” to refer to any language that is not signed, no matter whether it is repre- sented as text or audio, and no matter whether the discourse is formal (e.g. writing) or informal (e.g. dialogue). spoken languages. Recently, Yin et al. (2021) called for including signed languages in NLP re- search. We regard our shared task as a direct an- swer to this call. While WMT has a long history of shared tasks for spoken languages (Akhbardeh et al., 2021), this is the first time that signed lan- guages are included in a WMT shared task. Sign language translation requires processing visual information (such as video frames or human pose estimation) beyond the well-known paradigm of text-to-text machine translation (MT). As a con- sequence, solutions need to consider a combina- tion of Natural Language Processing (NLP) and computer vision (CV) techniques. In the field of sign language MT there is a gen- eral lack of suitable and freely available datasets and code. For this reason it was necessary for us to build and distribute novel training corpora and we also published reproducible …

Content truncated to 500 words