A Web Tool for Building Parallel Corpora of Spoken and Sign Languages

A Web Tool for Building Parallel Corpora of Spoken and Sign Languages Alex Becker∗, Fabio Kepler∗†, Sara Candeias‡ ∗UNIPAMPA - Federal University of Pampa – Alegrete/Brazil †L2F/INESC-ID – Lisbon/Portugal ‡Microsoft LDC – Lisbon/Portugal ∗alex@porthal.com.br, †fabio@kepler.pro.br, ‡t-sacand@microsoft.com Abstract In this paper we describe our work in building an online tool for manually annotating texts in any spoken language with SignWriting in any sign language. The existence of such tool will allow the creation of parallel corpora between spoken and sign languages that can be used to bootstrap the creation of efficient tools for the Deaf community. As an example, a parallel corpus between English and American Sign Language could be used for training Machine Learning models for automatic translation between the two languages. Clearly, this kind of tool must be designed in a way that it eases the task of human annotators, not only by being easy to use, but also by giving smart suggestions as the annotation progresses, in order to save time and effort. By building a collaborative, online, easy to use annotation tool for building parallel corpora between spoken and sign languages we aim at helping the development of proper resources for sign languages that can then be used in state-of-the-art models currently used in tools for spoken languages. There are several issues and difficulties in creating this kind of resource, and our presented tool already deals with some of them, like adequate text representation of a sign and many to many alignments between words and signs. Keywords: Sign language recognition, corpora creation, crowd-sourcing 1. Introduction Sign languages are the main way of communication in the Deaf community and with the listening population, and are almost always expressed in the form of visual signs and ex- pressions. There are over 200 distinct sign languages in the world, and about 70 million deaf people1. Some countries have an official sign language, like Brazil, where LIBRAS (Brazilian Sign Language, in Portuguese) is the second of- ficial language besides Portuguese. Brazil also has 5.7 mil- lion people with hearing impairment (of the Deaf, 2008; IBGE, 2000), with 1.7 million being sign language users. Unfortunately, as one could infer from the numbers above, many deaf do not know a sign language. Instead, they com- municate via gestures and by vocalizing (i.e., trying to ut- ter words). This lack of communication capabilities makes them prone to be highly dependent or to live isolated so- cially. Nevertheless, several deaf adults cannot read nor write a spoken language. When there is prelingual deafness, i.e., when a child loses hearing before beginning to speak, a sign language be- comes the child’s native language. This impairs their ability to acquire a spoken language (in its written form). And be- cause schools are not prepared and usually apply bad learn- ing methods, deaf children stay behind their colleagues in terms of knowledge development. They eventually reach the same level, but usually take longer. A relatively new form of expressing sign languages is the written form. There are several …

Content truncated to 500 words