Can Syntax Help? Improving an LSTM-based Sentence Compression Model for New Domains

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1385–1393 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1127 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1385–1393 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1127 Can Syntax Help? Improving an LSTM-based Sentence Compression Model for New Domains Liangguo Wang†∗, Jing Jiang∗, Hai Leong Chieu⋆, Chen Hui Ong⋆, Dandan Song†, Lejian Liao† chenwangliangguo@bit.edu.cn, jingjiang@smu.edu.sg {chaileon, ochenhui}@dso.org.sg, {sdd, liaolj}@bit.edu.cn † School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China ∗ School of Information Systems, Singapore Management University, Singapore ⋆ DSO National Laboratories, Singapore Abstract In this paper, we study how to improve the domain adaptability of a deletion-based Long Short-Term Memory (LSTM) neu- ral network model for sentence compres- sion. We hypothesize that syntactic in- formation helps in making such models more robust across domains. We propose two major changes to the model: using explicit syntactic features and introducing syntactic constraints through Integer Lin- ear Programming (ILP). Our evaluation shows that the proposed model works bet- ter than the original model as well as a tra- ditional non-neural-network-based model in a cross-domain setting. 1 Introduction Sentence compression is the task of compress- ing long, verbose sentences into short, concise ones. It can be used as a component of a text summarization system. Figure 1 shows two ex- ample input sentences and the compressed sen- tences written by human. The task has been stud- ied for almost two decades. Early work on this task mostly relies on syntactic information such as constituency-based parse trees to help decide what to prune from a sentence or how to re-write a sentence (Jing, 2000; Knight and Marcu, 2000). Recently, there has been much interest in apply- ing neural network models to solve the problem, where little or no linguistic analysis is performed except for tokenization (Filippova et al., 2015; Rush et al., 2015; Chopra et al., 2016). Although neural network-based models have achieved good performance on this task recently, they tend to suffer from two problems: (1) They require a large amount of data for training. For ex- ample, Filippova et al. (2015) used close to two In-domain Input: The southern Chinese city of Guangzhou has set up a special zone allowing foreign consulates to build per- manent offices and residences and avoid prohibitive local rents, the china daily reported Tuesday. Compressed (by human): Guangzhou opens new consulate area. Compressed (by machine): Guangzhou sets up special zone for foreign consulates. Out-of-domain Input: Wherever she was, she helped other loyal and flexi- ble wives cope. Compressed (by human): she helped other wives cope. Compressed (by machine): wives and flexible wives Figure 1: Examples of in-domain and out-of- domain results by a standard abstractive sequence- to-sequence model trained on the Gigaword cor- pus. The first input sentence comes from the Gi- gaword corpus while the second input sentence comes from the written news corpus …

Content truncated to 500 words