Character-based Thai Word Segmentation with Multiple Attentions

Proceedings of Recent Advances in Natural Language Processing, pages 264–273 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_031 264 Character-based Thai Word Segmentation with Multiple Attentions Thodsaporn Chay-intr, Hidetaka Kamigaito, Manabu Okumura Institute of Innovative Research, Tokyo Institute of Technology {chayintr@lr., kamigaito@lr., oku@}pi.titech.ac.jp Abstract Character-based word-segmentation models have been extensively applied to agglutinative languages, including Thai, due to their high performance. These models estimate word boundaries from a character sequence. How- ever, a character unit in sequences has no es- sential meaning, compared with word, sub- word, and character cluster units. We propose a Thai word-segmentation model that uses var- ious types of information, including words, subwords, and character clusters, from a char- acter sequence. Our model applies multiple at- tentions to refine segmentation inferences by estimating the significant relationships among characters and various unit types. The exper- imental results indicate that our model can outperform other state-of-the-art Thai word- segmentation models. 1 Introduction Thai running text has neither essential word de- limiters nor sentence periods. However, spaces are arbitrarily allowed to separate words, phrases, clauses, and sentences. These characteristics make word segmentation in Thai more difficult than in other languages, such as English, German, and Finnish, which have spaces and periods to identify word and sentence boundaries. Thai word segmen- tation can be categorized as a sequential-labeling task that assigns a word-boundary label to each character in a fine-grained tagging scheme such as BIES (beginning, inside, end, and singleton) (Xue, 2003), as shown in Figure 1. Neural-network models have been applied and perform well on character-based Thai word seg- mentation. Jousimo et al. (2017) applied bidirec- tional recurrent neural networks with gated recur- rent units, while Chormai et al. (2019) proposed an AttaCut, a convolutional-neural-network (CNN)- based model that mainly provides faster and more ม ี ๓ ค ว า ม ห ม า ย B E S B I I I I I I E Table 1: Thai word segmentation as a sequence labelling task on BIES taggging scheme. test 1 Thai มี ความหมาย ๓ ประการ ม ี ๓ ค ว า ม ห ม า ย B E S B I I I I I I E Table 1: Thai word segmentation as a sequence labelling task on BIES taggging scheme. มี ๓ ความหมาย “There are three meanings.” test 1 Thai มี ความหมาย ๓ ประการ Figure 1: Thai word segmentation as sequence-labeling task on BIES (beginning, inside, end, and singleton) tagging scheme accurate word inferring motivated by DeepCut (Kit- tinaradorn et al., 2019). Despite the fact that neural-network models with linguistic knowledge can perform almost perfectly, adapting models with pre-trained neural networks, such as word vectors and language models, is still useful (Shao et al., 2018). Seeha et al. (2020) pro- posed a transfer-learning approach for Thai word segmentation by using a pre-trained character lan- guage model for character-based word segmenta- tion. Although it exhibited state-of-the-art perfor- mance regarding Thai, it merely uses characters and does not use other information such as word and subword (Sennrich et al., 2016; Kudo, 2018). …

Content truncated to 500 words