CISLR: Corpus for Indian Sign Language Recognition

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10357 - 10366 December 7-11, 2022 ©2022 Association for Computational Linguistics CISLR: Corpus for Indian Sign Language Recognition Abhinav Joshi Ashwani Bhat Pradeep S Priya Gole Shreyansh Agarwal Shashwat Gupta Ashutosh Modi Indian Institute of Technology Kanpur (IIT-K) ajoshi@cse.iitk.ac.in, ashubhat44@gmail.com {spradeep20,priyagole20,shreyansha20,shashwatg20}@iitk.ac.in ashutoshm@cse.iitk.ac.in Abstract Indian Sign Language, though used by a di- verse community, still lacks well-annotated re- sources for developing systems that would en- able sign language processing. In recent years researchers have actively worked for sign lan- guages like American Sign Languages, how- ever, Indian Sign language is still far from data- driven tasks like machine translation. To ad- dress this gap, in this paper, we introduce a new dataset CISLR (Corpus for Indian Sign Lan- guage Recognition) for word-level recognition in Indian Sign Language using videos. The corpus has a large vocabulary of around 4700 words covering different topics and domains. Further, we propose a baseline model for word recognition from sign language videos. To han- dle the low resource problem in the Indian Sign Language, the proposed model consists of a prototype-based one-shot learner that leverages resource-rich American Sign Language to learn generalized features for improving predictions in Indian Sign Language. Our experiments show that gesture features learned in another sign language can help perform one-shot pre- dictions in CISLR. 1 Introduction Existing works in natural language processing have shown promising improvements in text classifica- tion, translation as well as generation in widely used spoken languages. However, sign language, on the other hand, still lacks sufficient resources for developing models using data-driven approaches, leading to low improvements in tasks like transla- tion and generation. One such low-resource sign language includes Indian Sign Language (ISL). As per the 2011 Indian Census, there are about 6 million deaf people in India (Wikipedia, 2022). According to Ethnologue (a reference publication documenting information about living languages of the world), ISL is the most widely used sign language in the world, and it is 151st most “spo- ken” language in the world (Ethnalogue, 2022). Figure 1: An example of the same signer showing dif- ferent signs for the same word “Buddhist.” Though the movement of the arm is similar, the hand gestures differ. However, there is a huge deficit of sign language interpreters e.g., according to the Government of India organization Indian Sign Language Research and Training Center (ISLRTC), there are only 300 certified sign language interpreters in India (http://islrtc.nic.in/). This has resulted not only in the widening of the communication bar- rier between the deaf community and the rest of the population but also has resulted in the very limited development of educational material for sign languages. On the technology side, there are lack of standard benchmarks for ISL resulting in low development and a lack of comparison of ma- chine learning based solutions for ISL, e.g., word recognition, translation, etc. In contrast, relatively speaking, other sign languages (e.g., American Sign Language (ASL), Deutsche Gebärdensprache (DGS)) have a …

Content truncated to 500 words