First WMT Shared Task on Sign Language Translation (WMT-SLT22)

First WMT Shared Task on Sign Language Translation (WMT-SLT22) Mathias M¨uller University of Zurich Sarah Ebling University of Zurich Eleftherios Avramidis DFKI Berlin Alessia Battisti University of Zurich Mich`ele Berger HfH Zurich Richard Bowden University of Surrey Annelies Braffort University of Paris-Saclay Necati Cihan Camg¨oz Meta Reality Labs Cristina Espa˜na-Bonet DFKI Saarbr¨ucken Roman Grundkiewicz Microsoft Zifan Jiang University of Zurich Oscar Koller Microsoft Amit Moryossef Bar-Ilan University Regula Perrollaz HfH Zurich Sabine Reinhard HfH Zurich Annette Rios University of Zurich Dimitar Shterionov Tilburg University Sandra Sidler-Miserez HfH Zurich Katja Tissi HfH Zurich Davy Van Landuyt European Union of the Deaf Abstract This paper is a brief summary of the First WMT Shared Task on Sign Language Translation (WMT-SLT22), a project partly funded by EAMT. The focus of this shared task is automatic translation between signed and spoken languages. Details can be found on our website1 or in the findings paper (M¨uller et al., 2022). 1 Project duration The project ran roughly from July 2021 (when the organizing commitee was assembled) to December 2022 (presentation of final results at WMT). 2 Description of the project This project entailed planning and realizing a WMT shared task on automatic translation be- tween signed and spoken2 languages. Recently, Yin et al. (2021) called for including signed lan- guages in natural language processing (NLP) re- search. We regard our shared task as a direct an- swer to this call. While WMT has a long history of shared tasks for spoken languages (Akhbardeh © 2023 The authors. This article is licensed under a Creative Commons 4.0 licence, no derivative works, attribution, CC- BY-ND. 1https://www.wmt-slt.com/ 2In this paper we use the word “spoken” to refer to any lan- guage that is not signed, no matter whether it is represented as text or audio, and no matter whether the discourse is formal (e.g. writing) or informal (e.g. dialogue). et al., 2021), this is the first time that signed lan- guages are included in a WMT shared task. The task is novel in the sense that it requires pro- cessing visual information (such as video frames or human pose estimation) beyond the well-known paradigm of text-to-text machine translation (MT). As a consequence, solutions need to consider a combination of NLP and computer vision (CV) techniques. The task featured two tracks, translating from Swiss German Sign Language (DSGS) to German and vice versa. 3 Objectives The project envisioned that there would be bene- fits both for Deaf sign language users and for the research community. For Deaf communities, the shared task aimed for better access to linguistic tools, including MT, in their native languages and also to improve recog- nition for sign languages. For the MT research community, our goal was to include sign languages in WMT shared tasks as a way of informing researchers about sign languages and boosting research on sign language translation. More concretely, we were looking to produce public benchmark data for MT systems, transla- tions by many state-of-the-art systems and judge- ments of translation quality …

Content truncated to 500 words