Neural Network Language Model for Chinese Pinyin Input Method Engine

Neural Network Language Model for Chinese Pinyin Input Method Engine Shen-Yuan Chen1,2, Rui Wang1,2 and Hai Zhao1,2∗ 1Center for Brain-Like Computing and Machine Intelligence, Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, China 2Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China chenshenyuan1992@sina.com, wangrui.nlp@gmail.com and zhaohai@cs.sjtu.edu.cn Abstract Neural network language models (NNLMs) have been shown to outperform traditional n- gram language model. However, too high computational cost of NNLMs becomes the main obstacle of directly integrating it into pinyin IME that normally requires a real-time response. In this paper, an efficient solution is proposed by converting NNLMs into back-off n-gram language models, and we integrate the converted NNLM into pinyin IME. Our exper- imental results show that the proposed method gives better decoding predictive performance for pinyin IME with satisfied efficiency. 1 Introduction Input method engine (IME) is the encoding method to input a variety of characters into computer or oth- er information processing and communication de- vices, such as mobile phone. People working with computer cannot live without IMEs. With the devel- opment and improvement of IMEs, people are pay- ing more and more attention to its efficiency and hu- manization. Since there are thousands of Chinese characters and only 26 English letters on standard keyboard, we cannot directly input the Chinese char- acters to the computer by simply hitting keys. A mapping or encoding from Chinese characters to En- glish letters is required, and IME is such a system software to do the mapping (Chen and Lee, 2000; Wu et al., 2009; Zhao et al., 2006). Among var- ious encoding schemes, most IMEs adopt Chinese ∗Corresponding author pinyin, which is the phonetic representation of Chi- nese characters through Latin (English) letters. The advantage of pinyin IMEs is that it only requires for knowledge of pinyin rules, which are known by most educated Chinese users. Being easy to learn and use, pinyin IME is becoming the first choice of more and more Chinese users. However, there are only under 500 pinyin sylla- bles in standard Chinese, mandarin, but over 6,000 common Chinese characters, bringing huge ambigu- ities for pinyin-to-characters mapping (Jia and Zhao, 2014; Xu and Zhao, 2012; Zhang et al., 2014). Mod- ern pinyin IMEs commonly use sentences-based de- coding method (Chen and Lee, 2000; Zhang et al., 2012), that is, generating a Chinese sentence accord- ing to a pinyin sequence for disambiguation. The decoding method usually works with the help of s- tatistic language model or other techniques. Back-off N-gram language models (BNLMs) (Chen and Goodman, 1996; Chen and Goodman, 1999; Stolcke, 2002a) have been broadly used for pinyin IME because of their simplicity and efficien- cy. Recently, continuous-space language models (CLSMs), especially neural network language mod- els (NNLMs) (Bengio et al., 2003; Schwenk, 2007; Le et al., 2010) are used in more and more natu- ral language processing tasks, and practically out- perform BNLMs in prediction accuracy. However, NNLMs are still out …

Content truncated to 500 words