An Empirical Comparison Between N-gram and Syntactic Language Models for Word Ordering

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 369–378, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. An Empirical Comparison Between N-gram and Syntactic Language Models for Word Ordering Jiangming Liu and Yue Zhang Singapore University of Technology and Design, 8 Somapah Road, Singapore, 487372 {jiangming liu, yue zhang}@sutd.edu.sg Abstract Syntactic language models and N-gram language models have both been used in word ordering. In this paper, we give an empirical comparison between N-gram and syntactic language models on word or- der task. Our results show that the quality of automatically-parsed training data has a relatively small impact on syntactic mod- els. Both of syntactic and N-gram mod- els can benefit from large-scale raw text. Compared with N-gram models, syntac- tic models give overall better performance, but they require much more training time. In addition, the two models lead to differ- ent error distributions in word ordering. A combination of the two models integrates the advantages of each model, achieving the best result in a standard benchmark. 1 Introduction N-gram language models have been used in a wide range of the generation tasks, such as machine translation (Koehn et al., 2003; Chiang, 2007; Galley et al., 2004), text summarization (Barzilay and McKeown, 2005) and realization (Guo et al., 2011). Such models are trained from large-scale raw text, capturing distributions of local word N- grams, which can be used to improve the fluency of synthesized text. More recently, syntactic language models have been used as a complement or alternative to N- gram language models for machine translation (Charniak et al., 2003; Shen et al., 2008; Schwartz et al., 2011), syntactic analysis (Chen et al., 2012) and tree linearization (Song et al., 2014). Compared with N-gram models, syntactic mod- els capture rich structural information, and can be more effective in improving the fluency of large constituents, long-range dependencies and over- all sentential grammaticality. However, Syntactic models require annotated syntactic structures for training, which are expensive to obtain manually. In addition, they can be slower compared to N- gram models. In this paper, we make an empirical compari- son between syntactic and N-gram language mod- els on the task of word ordering (Wan et al., 2009; Zhang and Clark, 2011a; De Gispert et al., 2014), which is to order a set of input words into a gram- matical and fluent sentence. The task can be re- garded as an abstract language modeling problem, although methods have been explored extending it for tree linearization (Zhang, 2013), broader text generation (Song et al., 2014) and machine trans- lation (Zhang et al., 2014). We choose the model of Liu et al.(2015) as the syntactic language model. There has been two main types of syntactic language models in the literature, the first being relatively more oriented to syntactic structure, without an explicit empha- sis on word orders (Shen et al., 2008; Chen et al., 2012). As a result, this type of syntactic language models are typically used jointly with N-gram …

Content truncated to 500 words