A Study on Implementation of Southern-Min Taiwanese Tone Sandhi System

Proceedings of PACLIC 19, the 19th Asia-Pacific Conference on Language, Information and Computation. A Study on Implementation of Southern-Min Taiwanese Tone Sandhi System Iun Un-gian Dept. of Computer Science and Info. Eng., National Taiwan Univ. No. 1, Sec. 4, Roosevelt Rd., Taipei 106, Taiwan d93001@csie.ntu.edu.tw Lau Kiat-gak Dept. of Computer Science and Info. Eng., National Taiwan Univ. No. 1, Sec. 4, Roosevelt Rd., Taipei 106, Taiwan kiatgak@gmail.com Li Sheng-an Dept. of Computer Science and Info. Eng., National Taiwan Univ. No. 1, Sec. 4, Roosevelt Rd., Taipei 106, Taiwan d93005@csie.ntu.edu.tw Kao Cheng-yan Dept. of Computer Science and Info. Eng., National Taiwan Univ. No. 1, Sec. 4, Roosevelt Rd., Taipei 106, Taiwan cykao@csie.ntu.edu.tw Abstract In the past two hundred years or so, a sizable corpus of Taiwanese text in Latin script has been accumulated. However, due to the political and historical situation of Taiwan, few people can read these materials at present. It is regrettable that the utilization of these plentiful materials is very low. This paper addresses problems raised by the Taiwanese tone sandhi system by describing a set of computational rules to approximate this system, as well as the results obtained from our implementation. Using the Taiwanese Latinization text as source, we take the sentence as the unit, translate every word into Chinese via a Taiwanese-Chinese dictionary, and obtain the POS information from the CKIP dictionary made by the CKIP group of the Academia Sinica. Using the POS data and tone sandhi rules we formulated based on linguistics, we then tag each syllable with its post-sandhi tone marker. Finally we implemented a Taiwanese tone sandhi processing system which takes a Latinized sentence as input and outputs the tone markers. We were able to obtain an accuracy rate of 97.56% and 88.90% with training and testing data, respectively. We analyze the sources of error for the purpose of future improvement. Keywords: written Taiwanese, tone sandhi system, Taiwanese latinization 1. Introduction 1.1.Background Taiwanese is often used in daily life in Taiwan, but written Taiwanese is less common by far; even so, the history of written Taiwanese stands at well over a century. (Tiun 2001) At present there are several dozen if not more than a hundred proposed phonetic and writing systems for Taiwanese. (Iun&Tiun 1999) The orthography adopted by this article is Peh-oe-ji (POJ, also known as Latinized Taiwanese or Church Latinized Taiwanese). Under the auspices of the National Museum of Taiwanese Literature, the Department of Taiwanese Literature of Cheng Kung University carried out a project titled 'The Collection and Cataloging of Peh-oe-ji Literature Data'. Although a lot of texts have already been lost due to the changing political situation, this project nevertheless revealed nearly 2,000 Peh-oe-ji books and periodicals, with publication sites spread over Taiwan, Xiamen, Shanghai, Guangzhou, Hong Kong, Singapore, Philippine, London, Japan, etc. The amount of publishing peaked in the 1950’s and 1960’s. (Iun&Tan-Ten 2005) The scope covers not only formally published books and periodicals but also non-published items such as personal letters and medical charts. Later on the …

Content truncated to 500 words