From Text to Sign Language: Exploiting the Spatial and Motioning Dimension

Proceedings of PACLIC 19, the 19th Asia-Pacific Conference on Language, Information and Computation. From Text to Sign Language: Exploiting the Spatial and Motioning Dimension Ji-Won Choi Computer Science Division, KAIST 373-1 Guseong-dong, Yuseong-gu, Daejeon 305-701, South Korea identity@nlp.kaist.ac.kr Hee-Jin Lee Computer Science Division, KAIST 373-1 Guseong-dong, Yuseong-gu, Daejeon 305-701, South Korea heejin@nlp.kaist.ac.kr Jong C. Park Computer Science Division, KAIST 373-1 Guseong-dong, Yuseong-gu, Daejeon 305-701, South Korea park@nlp.kaist.ac.kr Abstract In this paper, we address the problem of automatically converting information in the Korean language to one in a sign language as used in Korea. First, we discuss the differences between sign language and natural language, and in particular between the sign language in Korea and the Korean language. Then, we focus on issues that are relevant to the process of converting expressions in Korean into their counterparts in the sign language, including: 1) making explicit elided subjects of expressions in Korean, 2) omitting some expressions in Korean, and 3) reordering some expressions. We argue that it is important to utilize the spatial and motioning dimensionality of a sign language in order to minimize information loss and distortion. We also argue that the right decision to omit, or to merge some expressions in Korean plays a key role in exploiting this dimensionality. Finally, we present a system that converts sentences in Korean into corresponding animations in the sign language as proof of evidence for our claim. 1. Introduction The abundance of information in the present day, mostly in the form of written and spoken languages, has inevitably deepened the information divide among the people with different backgrounds. In this paper, we address the problems of automatically converting information in natural language to one in sign language, so as to lessen the information divide that people with different levels of hearing difficulty experience. For concreteness, we study the differences between the Korean language and the sign language as used in Korea. Earlier work in the field has mostly defined one-to-one relations between unit expressions in spoken and sign languages, along with simple translation rules. However, they suffer from the problem of not really reflecting the differences in the information space that the two types of language have. It is thus not surprising that an expression in spoken language becomes either unnecessarily verbose or severely restricted in its counterpart in sign language. In this paper, we argue that it is important to utilize the spatial and motioning dimensionality of a sign language to minimize information loss and distortion. For this reason, we also present a system that converts sentences in Korean into corresponding animations in the sign language, as proof of evidence for our claim. The rest of this paper is organized as follows. Section 2 describes other approaches to converting a natural language sentence into the corresponding sign language expression. Section 3 discusses the differences between sign language and natural language, Section 4 explores issues that arise when natural language sentences are converted into sign language expressions, using characteristics of the sign language …

Content truncated to 500 words