Sign Language Machine Translation and the Sign Language Lexicon: A Linguistically Informed Approach

Sign Language Machine Translation and the Sign Language Lexicon: A Linguistically In- formed Approach Irene%Murtagh% irene.murtagh@adaptcentre.ie0 Department0of0Informatics,0TU0Dublin,0Ireland0 Víctor%Ubieto%Nogales% victoremilio.ubieto@upf.edu0 Josep%Blat%% josep.blat@upf.edu0 Departament0de0Tecnologies0de0la0Informació0i0les0Comunicacions0Human-Com- puter0Interaction,0graphics0and0Educational0Technologies.0 ! Abstract Natural language processing and the machine translation of spoken language (speech/text) has benefitted from significant scientific research and development in recent times, rapidly advancing the field. On the other hand, computational processing and modelling of signed language has unfortunately not garnered nearly as much interest, with sign languages gener- ally being excluded from modern language technologies. Many deaf and hard-of-hearing in- dividuals use sign language on a daily basis as their first language. For the estimated 72 mil- lion deaf people in the world, the exclusion of sign languages from modern natural language processing and machine translation technology, aggravates further the communication barrier that already exists for deaf and hard-of-hearing individuals. This research leverages a linguis- tically informed approach to the processing and modelling of signed language. We outline current challenges for sign language machine translation from both a linguistic and a technical prespective. We provide an account of our work in progress in the development of sign lan- guage lexicon entries and sign language lexeme repository entries for SLMT. We leverage Role and Reference Grammar together with the Sign_A computational framework within this development. We provide an XML description for Sign_A, which is utilised to document SL lexicon entries together with SL lexeme repository entries. This XML description is also lev- eraged in the development of an extension to Bahavioural Markup Language, which will be used within this development to link the divide between the sign language lexicon and the avatar animation interface. 1. Introduction Sign Languages (SLs) are visual gestural languages articulated within a three-dimensional sign- ing space and have no written form (Murtagh, 2019a). Many deaf and hard-of-hearing individ- uals use SL on a daily basis as their first language. For the estimated 72 million deaf people in the world, the exclusion of sign languages from modern natural language processing and ma- chine translation technology, further aggravates the communication barrier that already exists for deaf and hard-of-hearing individ- uals (Allen, 2013). We outline our research work in progress in the development of a SL lexicon architecture, including SL lexicon entries and SL lexeme repository entries for a sign language machine translation (SLMT) system. We provide some background information on the Role and Reference Grammar (RRG) and the Sign_A framework, which are leveraged within this development. We discuss the XML specification for the Sign_A computational framework, Proceedings of the 15th Biennial Conference of the Association for Machine Translation in the Americas Orlando, USA, September 12-16, 2022. Volume 1: Research Track 240 which we leverage to define SL lexicon entires and SL lexeme repository entries. We also dis- cuss the extension to the specification for Behavioural Markup Language in the development of a planner for SL translation. 2. SignON Project We draw here on work we are engaged in for the Horizon 2020 funded SignON project, which seeks to create a service that translates …

Content truncated to 500 words