Incorporating Word Attention into Character-Based Word Segmentation

Proceedings of NAACL-HLT 2019, pages 2699–2709 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2699 Incorporating Word Attention into Character-Based Word Segmentation Shohei Higashiyama1, Masao Utiyama1, Eiichiro Sumita1, Masao Ideuchi2, Yoshiaki Oida2, Yohei Sakamoto2, Isaac Okada2 1National Institute of Information and Communications Technology, Kyoto, Japan 2FUJITSU LIMITED, Tokyo, Japan {shohei.higashiyama, mutiyama, eiichiro.sumita}@nict.go.jp, {ideuchi.masao,oida.yoshiaki,yohei.sakamoto,isaac-okada} @jp.fujitsu.com Abstract Neural network models have been actively ap- plied to word segmentation, especially Chi- nese, because of the ability to minimize the effort in feature engineering. Typical seg- mentation models are categorized as character- based, for conducting exact inference, or word-based, for utilizing word-level informa- tion. We propose a character-based model uti- lizing word information to leverage the advan- tages of both types of models. Our model learns the importance of multiple candidate words for a character on the basis of an atten- tion mechanism, and makes use of it for seg- mentation decisions. The experimental results show that our model achieves better perfor- mance than the state-of-the-art models on both Japanese and Chinese benchmark datasets.1 1 Introduction Word segmentation is the first step of natural lan- guage processing (NLP) for most East Asian lan- guages, such as Japanese and Chinese. In recent years, neural network models have been widely applied to word segmentation, especially Chinese, because of their ability to minimize the effort in feature engineering. These models are catego- rized as character-based or word-based. Word- based models (Zhang et al., 2016; Cai and Zhao, 2016; Cai et al., 2017; Yang et al., 2017) directly segment a character sequence into words and can easily achieve the benefits of word-level informa- tion. However, these models cannot usually con- duct exact inference because of strategies, such as beam-search decoding and constraints of max- imum word length, which are necessary as the number of candidate segmentations increases ex- ponentially with the sentence length. On the other hand, character-based models (Zheng et al., 2013; 1 We have released our code at https://github. com/shigashiyama/seikanlp. Mansur et al., 2013; Pei et al., 2014; Chen et al., 2015a) treat word segmentation as sequence label- ing. These models typically predict optimal label sequences while considering adjacent labels. Limited efforts have been devoted to leveraging the advantages of both types of models, such as utilizing word information and conducting exact inference, which are complementary characteris- tics. In particular, the candidate word information for a character is beneficial to disambiguate word boundaries because a character in the sentence has multiple candidate words that contain the charac- ter. For example, there are three or four candidate words for characters x3, x4 and x5 in a sentence x1:5 in Figure 1. A feasible solution to develop a model with both characteristics is to incorporate word information into a character-based frame- work. An example of such work is that of Wang and Xu (2017). They concatenated embeddings of a character and candidate words and used it in their convolutional neural network (CNN)-based model. They treated candidate words equivalently, although the plausibility of a …

Content truncated to 500 words