Rosetta-LSF: an Aligned Corpus of French Sign Language and French for Text-to-Sign Translation

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4955–4962 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4955 Rosetta-LSF: an Aligned Corpus of French Sign Language and French for Text-to-Sign Translation ´Elise Bertin-Lem´ee1, Annelies Braffort2, Camille Challant2, Claire Danet2, Boris Dauriac3, Michael Filhol2, Emmanuella Martinod2, J´er´emie Segouat4 1SYSTRAN, 5 rue Feydeau, Paris, elise.bertinlemee@systrangroup.com 2LISN, Universit´e Paris-Saclay, bˆat 507 rue du Belved`ere, Orsay, {annelies.braffort, camille.challant, claire.danet, michael.filhol, emmanuella.martinod}@lisn.upsaclay.fr 3R&D MocapLab, 70 rue du Landy, 93300 Aubervilliers, boris.dauriac@mocaplab.com 4STILS / CLLE, Universit´e Jean Jaur`es, 5 all´ee Antonio Machado, 31058 Toulouse, jeremie.segouat@univ-tlse2.fr Abstract This article presents a new French Sign Language (LSF) corpus called Rosetta-LSF. It was created to support future studies on the automatic translation of written French into LSF, rendered through the animation of a virtual signer. An overview of the field highlights the importance of a quality representation of LSF. In order to obtain quality animations understandable by signers, it must surpass the simple “gloss transcription” of the LSF lexical units to use in the discourse. To achieve this, we designed a corpus composed of four types of aligned data, and evaluated its usability. These are: news headlines in French, translations of these headlines into LSF in the form of videos showing animations of a virtual signer, gloss annotations of the “traditional” type—although including additional information on the context in which each gestural unit is performed as well as their potential for adaptation to another context—and AZee representations of the videos, i.e. formal expressions capturing the necessary and sufficient linguistic information. This article describes this data, exhibiting an example from the corpus. It is available online for public research. Keywords: Sign Language, machine translation, example-based translation, synthesis 1. Introduction This article presents a new French Sign Language (LSF) corpus called Rosetta-LSF (Dauriac, 2022). It has been designed with the main objective of allowing exploratory studies on the automatic translation of written French into LSF, output through virtual signer animation. A first study of this type is described in (Bertin-Lem´ee et al., 2022) for the translation part and in (Dauriac et al., 2022) for the an- imation part. The objective is to test an example-based ap- proach, leveraging several types of aligned data. The following section (section 2) specifies the framework of this study and explains a few choices made. Sections 3 and 4 respectively describe how the corpus was built and extended. Finally, section 5 describes in more detail an ex- ample taken from this corpus. 2. Text-to-Sign translation The problem of automatically generating Sign Language (SL) from written language is one of translation, with text as the source format, in our case in French, and video or 3D animation as the target format, French Sign Language (LSF) for us. The meaning of the utterance must be pre- served. In line with the methodology of human translators and interpreters (with the exception of Sign Language inter- preters) to preferentially be native in the target language to maximise the resulting fluency, …

Content truncated to 500 words