Crowdsourcing Kazakh-Russian Sign Language: FluentSigners-50

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 2541–2547 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 2541 Crowdsourcing Kazakh-Russian Sign Language: FluentSigners-50 M. Mukushev1, A. Kydyrbekova1, A. Imashev1, V. Kimmelman2, A. Sandygulova1 1Department of Robotics and Mechatronics, School of Engineering and Digital Sciences Nazarbayev University, Nur-Sultan, Kazakhstan 2Department of Linguistic, Literary, and Aesthetic Studies University of Bergen, Bergen, Norway {mmukushev, aigerim.kydyrbekova, alfarabi.imashev, anara.sandygulova}@nu.edu.kz vadim.kimmelman@uib.no Abstract This paper presents the methodology we used to crowdsource a data collection of a new large-scale signer independent dataset for Kazakh-Russian Sign Language (KRSL) created for Sign Language Processing. By involving the Deaf community throughout the research process, we firstly designed a research protocol and then performed an efficient crowdsourcing campaign that resulted in a new FluentSigners-50 dataset. The FluentSigners-50 dataset consists of 173 sentences performed by 50 KRSL signers for 43,250 video samples. Dataset contributors recorded videos in real-life settings on various backgrounds using various devices such as smartphones and web cameras. Therefore, each dataset contribution has a varying distance to the camera, camera angles and aspect ratio, video quality, and frame rates. Additionally, the proposed dataset contains a high degree of linguistic and inter-signer variability and thus is a better training set for recognizing a real-life signed speech. FluentSigners-50 is publicly available at https://krslproject.github.io/fluentsigners-50/ Keywords: dataset, sign language processing, deaf, crowdsourcing 1. Introduction Natural languages utilized largely by deaf populations across the world are known as sign languages. In sign languages, meaning is expressed by a series of mo- tions involving the hands, torso, arms, head, and face. Sign languages, like spoken languages, contain several levels of linguistic structure, such as phonology, mor- phology, syntax, semantics, and pragmatics. Over 300 different sign languages have been recognized thus far (Koller, 2020). One of the main challenges, as Bragg et al. (2019) cor- rectly point out in their review of research in the field of Sign Language Processing (SLP), is related to signifi- cant limitations of public sign language datasets that limit the power and generalizability of recognition sys- tems trained on them. SLP is a research area focusing on sign language recognition, generation, and transla- tion. Apart from obvious dataset limitations such as the size of the vocabulary (due to expensive recording and annotation processes), most datasets only contain iso- lated signs (such as MS-ASL (Joze and Koller, 2018) and Devisign (Chai et al., 2014)), which are insuffi- cient for most real-world use cases that require natural signing (continuous and spontaneous) and training on complete sentences and longer utterances (Bragg et al., 2019). Another disadvantage of most presently used datasets is the lack of environmental heterogeneity since they are often recorded in the same setting(s) and have just one vocabulary domain, resulting in overfitting when applied to models that are architecturally more complex (Koller et al., 2016). As a result, several Continuous Sign Language Recognition (CSLR) tech- niques concentrate solely on cropped hands to make the task easier (Bragg et al., 2019). It …

Content truncated to 500 words