An Operation Network for Abstractive Sentence Compression

Proceedings of the 27th International Conference on Computational Linguistics, pages 1065–1076 Santa Fe, New Mexico, USA, August 20-26, 2018. 1065 An Operation Network for Abstractive Sentence Compression Naitong Yu Jie Zhang∗ Minlie Huang† Xiaoyan Zhu State Key Laboratory of Intelligent Technology and Systems Tsinghua National Laboratory for Information Science and Technology Dept. of Computer Science and Technology, Tsinghua University, Beijing, PR China ∗Microsoft Search Technology Center ynt12@mails.tsinghua.edu.cn zhanjie@microsoft.com aihuang@tsinghua.edu.cn zxy-dcs@tsinghua.edu.cn Abstract Sentence compression condenses a sentence while preserving its most important contents. Delete-based models have the strong ability to delete undesired words, while generate-based models are able to reorder or rephrase the words, which are more coherent to human sen- tence compression. In this paper, we propose Operation Network, a neural network approach for abstractive sentence compression, which combines the advantages of both delete-based and generate-based sentence compression models. The central idea of Operation Network is to model the sentence compression process as an editing procedure. First, unnecessary words are deleted from the source sentence, then new words are either generated from a large vocabulary or copied directly from the source sentence. A compressed sentence can be obtained by a series of such edit operations (delete, copy and generate). Experiments show that Operation Network outperforms state-of-the-art baselines. 1 Introduction Sentence compression is the natural language generation (NLG) task of condensing a sentence while preserving its most important contents. It can also be viewed as a sentence-level summarization task. With the rapid growth of the web contents in recent years, summarization techniques such as sentence compression are becoming more and more important since these techniques can greatly reduce the infor- mation overload on the web. Sentence compression can benefit a wide range of applications, especially those on mobile devices which have restricted screen spaces. Sentence compression models can be broadly classified into two categories: delete-based models and abstractive models. Delete-based ap- proaches remove unimportant words from the source sentence and generate a shorter sentence by stitch- ing the remaining fragments together. On the contrary, abstractive models consider operations beyond word deletion, such as reordering, substitution and insertion. Abstractive sentence compression models produce a reform of the source sentence from scratch, thus the results produced by abstractive sentence compression models are more expressive. Obviously, abstractive sentence compression is much harder than delete-based sentence compression, since it needs deeper understanding of the source sentence. Delete-based sentence compression treats the task as a word deletion problem: given an input source sentence x = x1, x2, ..., xn (where xi stands for the ith word in the sentence x), the goal is to produce a target sentence by removing any subset of words in the source sentence x (Knight and Marcu, 2002). Delete-based sentence compression has been widely explored across different modeling paradigms, such as noisy-channel model (Knight and Marcu, 2002; Turner and Charniak, 2005), large-margin learn- ing (McDonald, 2006; Cohn and Lapata, 2007), integer linear programming (Clarke and Lapata, 2008) and variational auto-encoder (Miao and Blunsom, 2016). In delete-based sentence compression models, only delete …

Content truncated to 500 words