Structure-Infused Copy Mechanisms for Abstractive Summarization

Proceedings of the 27th International Conference on Computational Linguistics, pages 1717–1729 Santa Fe, New Mexico, USA, August 20-26, 2018. 1717 Structure-Infused Copy Mechanisms for Abstractive Summarization Kaiqiang Song Computer Science Dept. University of Central Florida Orlando, FL 32816, USA kqsong@knights.ucf.edu Lin Zhao Research and Tech. Center Robert Bosch LLC Sunnyvale, CA 94085, USA lin.zhao@us.bosch.com Fei Liu Computer Science Dept. University of Central Florida Orlando, FL 32816, USA feiliu@cs.ucf.edu Abstract Seq2seq learning has produced promising results on summarization. However, in many cases, system summaries still struggle to keep the meaning of the original intact. They may miss out im- portant words or relations that play critical roles in the syntactic structure of source sentences. In this paper, we present structure-infused copy mechanisms to facilitate copying important words and relations from the source sentence to summary sentence. The approach naturally combines source dependency structure with the copy mechanism of an abstractive sentence summarizer. Experimental results demonstrate the effectiveness of incorporating source-side syntactic infor- mation in the system, and our proposed approach compares favorably to state-of-the-art methods. 1 Introduction Recent years have witnessed increasing interest in abstractive summarization. The systems seek to con- dense source texts to summaries that are concise, grammatical, and preserve the important meaning of the original texts (Nenkova and McKeown, 2011). The task encompasses a number of high-level text operations, e.g., paraphrasing, generalization, text reduction and reordering (Jing and McKeown, 1999), posing a considerable challenge to natural language understanding. Src A Mozambican man suspect of murdering Jorge Microsse, director of Maputo central prison, has escaped from the city’s police headquarters, local media reported on Tuesday. Ref Mozambican suspected of killing Maputo prison director escapes Sys mozambican man arrested for murder Src An Alaska father who was too drunk to drive had his 11-year-old son take the wheel, authorities said. Ref Drunk Alaska dad has 11 year old drive home Sys alaska father who was too drunk to drive Table 1: Example source sentences, reference and system summaries produced by a neural attentive seq-to-seq model. System summaries fail to preserve summary-worthy content of the source (e.g., main verbs) despite their syntactic importance. The sequence-to-sequence learning paradigm has achieved remarkable success on abstractive sum- marization (Rush et al., 2015; Nallapati et al., 2016; See et al., 2017; Paulus et al., 2017). While the results are impressive, individual system summaries can appear unreliable and fail to preserve the mean- ing of the source texts. Table 1 presents two examples. In these cases, the syntactic structure of source sentences is relatively rare but perfectly normal. The first sentence contains two appositional phrases (“suspect of murdering Jorge Microsse,” “director of Maputo central prison”) and the second sentence has a relative clause (“who was too drunk to drive”), both located between the subject and the main verb. The system, however, fails to identify the main verb in both cases; it instead chooses to focus on the first few words of the source sentences. We observe that rare syntactic constructions of the source can pose problems …

Content truncated to 500 words