Subword Encoding in Lattice LSTM for Chinese Word Segmentation

Proceedings of NAACL-HLT 2019, pages 2720–2725 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2720 Subword Encoding in Lattice LSTM for Chinese Word Segmentation Jie Yang♠♣, Yue Zhang♦, Shuailong Liang▲ ♠Brigham and Women’s Hospital. Boston, USA ♣Harvard Medical School, Harvard University. Boston, USA ♦School of Engineering, Westlake University. Hangzhou, China ▲Singapore University of Technology and Design. Singapore jieynlp@gmail.com yue.zhang@wias.org.cn shuailong liang@mymail.sutd.edu.sg Abstract We investigate subword information for Chi- nese word segmentation, by integrating sub word embeddings trained using byte-pair en- coding into a Lattice LSTM (LaLSTM) net- work over a character sequence. Experiments on standard benchmark show that subword in- formation brings significant gains over strong character-based segmentation models. To our knowledge, this is the first research on the ef- fectiveness of subwords on neural word seg- mentation. 1 Introduction Chinese word segmentation (CWS) is a traditional NLP task (Sproat et al., 1996), the features for which have been a central research topic. Statisti- cal methods consider characters (Xue et al., 2003), subwords (Zhang et al., 2006), and words (Zhang and Clark, 2007) as input features. Among these, both characters (Chen et al., 2015a) and words (Zhang et al., 2016; Cai and Zhao, 2016; Yang et al., 2017) have also shown useful in recent neu- ral models. However, how to utilize the subword features in neural networks has not been investi- gated yet. In this paper, we fill this gap by proposing a subword-based neural word segmentor, by inte- grating two strands of works: the byte pair en- coding (BPE) algorithm (Gage, 1994) and the lat- tice LSTM structure (Zhang and Yang, 2018). The BPE algorithm constructs a subword list from raw data and lattice LSTM introduces subwords into character LSTM representation. In partic- ular, our baseline is a BiLSTM-CRF segmentor (Chen et al., 2015b) and we replace LSTM with lattice LSTM using subwords to encode character composition information. Our code1 is based on NCRF++ (Yang and Zhang, 2018). 1Our code is released at https://github.com/ jiesutd/SubwordEncoding-CWS. 中 Center 国 Nation 科 Science 学 Academy 院 Institute 院 Institute 士 Fellow Character Sequence: Matched Words/Subwords: 中国 Chinese 科学 Science 学院 Academy 科学院 Academy of Science 院士 Fellow Gold word Sequence: 中国 Chinese 科学院 Academy of Science 院士 Fellow Figure 1: Segmentation with ambiguous words. Compared with character-based neural seg- mentors, our model can utilize abundant char- acter combination (subword) information, which is effective to disambiguate charac- ters. For example, in Figure 1, the subword “学 院(Academy)” ensures that the charac- ter “学” means “Academy(noun)” rather than “study(verb)”. Compared with the word-based neural models (Zhang et al., 2016; Cai and Zhao, 2016), ambiguous subwords in a context can pro- vide additional information for disambiguation. For instance, the subword “科学院(Academy of Sciences)” and “学院(Academy)” can be useful in determining the correct segmentation, which is “科学院/(Academy of Sciences/)”. To our knowledge, we are the first to use sub- words in a neural network segmentor. We inves- tigate the contributions of subword lexicons and their pretrained embeddings through controlled experiments. …

Content truncated to 500 words