Chinese Long and Short Form Choice Exploiting Neural Network Language Modeling Approaches

Chinese Long and Short Form Choice Exploiting Neural Network Language Modeling Approaches Lin Li Utrecht University Utrecht, the Netherlands Qinghai Normal University Xining, China l.li1@uu.nl Kees van Deemter Utrecht University Utrecht, the Netherlands c.j.vandeemter@uu.nl Denis Paperno Utrecht University Utrecht, the Netherlands d.paperno@uu.nl Abstract Lexicalisation is one of the most challenging tasks of Natural Language Generation (NLG). This paper presents our work in choosing between long and short forms of elastic words in Chinese, which is a key aspect of lexicalisation. Long and short forms is a highly frequent linguistic phenomenon in Chinese such as 老虎-虎 (laohu-hu, tiger). The choice of long and short form task aims to properly choose between long and short form for a given context to producing high-quality Chinese. We tackle long and short form choice as a word prediction question with neural network language modeling approaches because of their powerful language representation capa- bility. In this work, long and short form choice models based on the-state-of-art Neural Network Language Models (NNLMs) have been built, and a classical n-gram Language Model (LM) is constructed as a baseline system. A well-designed test set is constructed to evaluate our models, and results show that NNLMs-based models achieve significantly improved performance than the baseline system. 1 Introduction The long and short form of an elastic word refers to words have different word length (i.e. number of syllables) but share at least one identical word meaning such as 丢失-丢 (diushi-diu, lose). Duanmu(Duanmu, 2013) points out that as high as 80% percent of Chinese words has both long and short forms, therefore Chinese speakers need to make the choice between long and short forms during daily communication. Like human speakers and writers, the long and short form choice task also needs to be carefully resolved for various domain including Natural Language Generation(Inkpen and Hirst, 2004), Machine Translation(Nguyen and Chiang, 2017), and Style Transfer(Fu et al., 2018). In this work, we focus on long and short forms that share at least one same word meaning and one same morpheme, but compose of different number of syllables. The long and short form choice task is formulated as Fill-in-the-blank (FITB) task(Inkpen and Hirst, 2004; Zweig et al., 2012), whose goal is to select a missing word for a sentence from a set of candidates. A FITB example used in this work is shown in Table 1. Sentence Long Form Short Form 她去日本旅游时，必逛各种免税 ______。 (1) 商店 (2) 店 When travels to Japan, she must go to duty free_____. (1) shop (2) shop Table 1: A long and short form choice FITB question example. The lexical choice is difficult in the context of long and short forms for most language process- ing systems due to the identical word sense leading to their preceding and subsequent contexts are too similar to providing distinguishing information. To address this problem, we investi- gate in learning language representation by LMs to making elegant choice of long and short Proceedi ngs of t he 19t h Chi na Nat i onal Conf …

Content truncated to 500 words