Towards a Multi-Objective Corpus for Vietnamese Language

TOWARDS A MULTI-OBJECTIVE CORPUS FOR VIETNAMESE LANGUAGE Vu Hai Quan, Pham Nam Trung, Nguyen Duc Hoang Ha, Huynh Bao Toan, Le Hoai Bac, Hoang Kiem Vietnam Natitional Universiy at HCMCT 227 Nguyen Van Cu, HoChiMinh city,Vietnam vhquan@fit.hcmuns.edu.vn Abstract Today, corpus plays an important role in development and evaluation language and speech technologies, such as part of speech tagging, parsing, word sense disambiguation, text categorization, named entity classification, information extraction, question answering, structure discovery (clustering), speech recognition and machine translation systems, etc. One can exploit valuable statistical parameters taken from corpus to train and evaluate those systems. Developing such a corpus has been a challenging work in context that a huge data needed to be processed and annotated. In this paper we first represent our developing method for a multi- objective Vietnamese language corpus, namely VnCorpus, together with the description of various kinds of sources from which we have used to build up this database. It then goes on to describe some first experiences in using this corpus for the segmentation of sentences into Vietnamese words and for the recognition of Vietnamese continuous speech. Upon completion the corpus will constitute a valuable resource for research in the fields of computational linguistics, language and speech technologies. 1� Introduction Vietnamese speech has been created approximately 4000 years, closely related with Indo-European languages. Today there are around 80.000.000 people using this language. The main feature which makes it differ from Western languages is that it belongs among the group of mono - syllable languages. That means it never changes its morphology. In order to express grammatical sense we usually use means of the outside word as grammatical words, order words, etc. The other important feature that makes it differ from Eastern language is that it uses extended Latin based symbols. In Vietnamese language, the basic is "tiéng". There are totally around 8000 "tiéng" found in Vietnamese modern language [1]. For speech, in the complete form, "tiêng" has following model: (as shown in Fig 1) Tone initial sound Syllable inter-sound main sound final sound Fig. 1 Structure of "tiing" In some cases, "ti&ig" can be appeared without final sound, inter-sound, or initial sound. That means main sound and tone are the major components from which "ti8ng" is formed. There are 22 initial sounds, 14 main sounds and 10 final sounds coordinating with 6 tones. In order to distinguish "tiéng", initial sound, inter-sound, main sound and final sound are used. In cases that all of them are the same, tones are used. An example of "tiéne and tones are given in Fig. 2 and Fig. 3. 416 mag 1...1 1; .14 initial sound� main sound Ui t Fig. 2. Structure of Vietnamese "tieng" "BO" wave form 1 king fundamental frequency tract Fig 3. The different tones of Vietnamese For writing system, "tiêng" usually consists of two main components: consonant (corresponding to initial sound) and syllable (corresponding to inter-sound, main sound and final sound) coordinating with accent in the accent set (corresponding to tones). There are totally …

Content truncated to 500 words