Unsupervised Question Answering via Answer Diversifying

Proceedings of the 29th International Conference on Computational Linguistics, pages 152–162 October 12–17, 2022. 152 Character Jacobian: modeling Chinese character meanings with deep learning model Yu-Hsiang Tseng Graduate Institute of Linguistics National Taiwan University seantyh@gmail.com Shu-Kai Hsieh Graduate Institute of Linguistics National Taiwan University shukaihsieh@ntu.edu.tw Abstract Compounding, a prevalent word-formation process, presents an interesting challenge for computational models. Indeed, the re- lations between compounds and their con- stituents are often complicated. It is par- ticularly so in Chinese morphology, where each character is almost simultaneously bound and free when treated as a mor- pheme. To model such word-formation process, we propose the Notch (Nonlinear Transformation of Character embeddings) model and the character Jacobians. The Notch model first learns the non-linear rela- tions between the constituents and words, and the character Jacobians further de- scribe the character’s role in each word. In a series of experiments, we show that the Notch model predicts the embeddings of the real words from their constituents and helps account for the behavioral data of the pseudowords. Moreover, we also demonstrated that character Jacobians re- flect the characters’ meanings. Taken to- gether, the Notch model and character Ja- cobians may provide a new perspective on studying the word-formation process and morphology with modern deep learning. 1 Introduction Recent years have witnessed a growing interest in modeling the internal semantic dynamics of compounds. Indeed, compounding is some- times argued as a language universal, and it is claimed to be protolinguistic fossils which modern languages frequently elaborate. Com- pounds are usually loosely defined as form- ing words with two independent words, such as blackboard and pineapple (Libben, 2014; Bauer, 2009; Jackendoff, 2002). It is apparent that the number of potential combinations is already enormous, even in this simplest form of the two-constituent compound. The productivity of compounding is par- ticularly evident in Chinese word-formation. Due to the debating nature of Chinese word- hood, there is a vague boundary between word- forming affix and bound root when treating a Chinese character (字 zì) as a morpheme (Huang et al., 2017; Hsieh et al., 2018; Tseng et al., 2020). There is considerable flexibility in Chinese characters, where morphemes can be joined with one another either preceding or following them. For example, the following four words all start with the characters which are the ends of the previous one: 長老 zhǎng lǎo “elder”, 老師 lǎo shī “teacher”, 師範 shī fàn “teacher-training”, 範圍 fàn wéi “area”. The versatility of characters leads Hoosain (1992) to describe Chinese text as “a continuous pa- rade of meaningful individual characters (mor- pheme)” (see Packard (2000) for a complete introduction in Chinese morphology). However, the productivity of Chinese char- acters is hard to capture by a computational model. A traditional natural language pipeline starts with word segmentation, which removes the sublexical cues (characters) in the first step. Even for the later distributional seman- tic model with sub-word information, such as FastText (Bojanowski et al., 2016), it cannot accommodate the different meanings carried by individual Chinese characters. Recent …

Content truncated to 500 words