Exploiting Word Internal Structures for Generic Chinese Sentence Representation

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 298–303 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Exploiting Word Internal Structures for Generic Chinese Sentence Representation Shaonan Wang1,2, Jiajun Zhang1,2, Chengqing Zong1,2,3 1 National Laboratory of Pattern Recognition, CASIA, Beijing, China 2 University of Chinese Academy of Sciences, Beijing, China 3 CAS Center for Excellence in Brain Science and Intelligence Technology, Shanghai, China {shaonan.wang,jjzhang,cqzong}@nlpr.ia.ac.cn Abstract We introduce a novel mixed character- word architecture to improve Chinese sen- tence representations, by utilizing rich se- mantic information of word internal struc- tures. Our architecture uses two key s- trategies. The first is a mask gate on char- acters, learning the relation among char- acters in a word. The second is a max- pooling operation on words, adaptively finding the optimal mixture of the atom- s.ic and compositional word representation- Finally, the proposed architecture is applied to various sentence composition models, which achieves substantial perfor- mance gains over baseline models on sen- tence similarity task. 1 Introduction To understand the meaning of a sentence is a pre- requisite to solve many natural language process- ing problems. Obviously, this requires a good rep- resentation of the meaning of a sentence. Recent- ly, neural network based methods have shown ad- vantage in learning task-specific sentence repre- sentations (Kalchbrenner et al., 2014; Tai et al., 2015; Chen et al., 2015a; Cheng and Kartsaklis, 2015) and generic sentence representations (Le and Mikolov, 2014; Hermann and Blunsom, 2014; Kiros et al., 2015; Kenter et al., 2016; Wang et al., 2017). To learn generic sentence representation- s that perform robustly across tasks as effective as word representations, Wieting et al. (2016b) proposes an architecture based on the supervision from the Paraphrase Database (Ganitkevitch et al., 2013). Despite the fact that Chinese has unique word internal structures, there is no work focusing on learning generic Chinese sentence representation- Atomic word: Compositional word: 搭乘 出租车 到 虹桥 机场 搭-乘 出-租-车 到 虹桥 机-场 Final word embedding: Atomic word: Compositional word: 搭乘 出租车 到 虹桥 机场 搭-乘 出-租-车 到 虹桥 机-场 Final word: 搭乘 出租车 到 虹桥 机场 Atomic(Full-form): Compositional: 搭乘 出租车 到 虹桥 机场 搭-乘 出-租-车 到 虹桥 机-场 Final word: 搭乘 出租车 到 虹桥 机场 Figure 1: An example sentence that consists of five words as “搭乘(take) 出租车(taxi) 到(to) 虹 桥(Hongqiao) 机场(airport)”. Most of these word- s are compositional, namely word “搭乘” consists of characters “搭(take)” and “乘(ride)”, word “出 租车” constitutes characters “出(out)”, “租(rent)” and “车(car)”, and word “机场” is composed of characters “机(machine)” and “场(field)”. The color depth represents (1) contributions of each character to the compositional word meaning, and (2) contributions of the atomic (which ignore in- ner structures) and compositional word to the final word meaning. The deeper color means more con- tributions. s. In contrast to English, Chinese characters con- tain rich information and are capable of indicat- ing semantic meanings of words. As illustrated in Figure 1, the internal structures of Chinese word- s express two characteristics: …

Content truncated to 500 words