Neural Headline Generation on Abstract Meaning Representation

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1054–1059, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Neural Headline Generation on Abstract Meaning Representation Sho Takase† Jun Suzuki‡ Naoaki Okazaki† Tsutomu Hirao‡ Masaaki Nagata‡ Graduate School of Information Sciences, Tohoku University† NTT Communication Science Laboratories, NTT Corporation‡ {takase, okazaki}@ecei.tohoku.ac.jp {suzuki.jun, hirao.tsutomu, nagata.masaaki}@lab.ntt.co.jp Abstract Neural network-based encoder-decoder mod- els are among recent attractive methodologies for tackling natural language generation tasks. This paper investigates the usefulness of struc- tural syntactic and semantic information ad- ditionally incorporated in a baseline neural attention-based model. We encode results ob- tained from an abstract meaning representa- tion (AMR) parser using a modified version of Tree-LSTM. Our proposed attention-based AMR encoder-decoder model improves head- line generation benchmarks compared with the baseline neural attention-based model. 1 Introduction Neural network-based encoder-decoder models are cutting-edge methodologies for tackling natural lan- guage generation (NLG) tasks, i.e., machine trans- lation (Cho et al., 2014), image captioning (Vinyals et al., 2015), video description (Venugopalan et al., 2015), and headline generation (Rush et al., 2015). This paper also shares a similar goal and moti- vation to previous work: improving the encoder- decoder models for natural language generation. There are several directions for enhancement. This paper respects the fact that NLP researchers have expended an enormous amount of effort to develop fundamental NLP techniques such as POS tagging, dependency parsing, named entity recognition, and semantic role labeling. Intuitively, this structural, syntactic, and semantic information underlying in- put text has the potential for improving the quality of NLG tasks. However, to the best of our knowledge, there is no clear evidence that syntactic and seman- tic information can enhance the recently developed encoder-decoder models in NLG tasks. To answer this research question, this paper pro- poses and evaluates a headline generation method based on an encoder-decoder architecture on Ab- stract Meaning Representation (AMR). The method is essentially an extension of attention-based sum- marization (ABS) (Rush et al., 2015). Our pro- posed method encodes results obtained from an AMR parser by using a modified version of Tree- LSTM encoder (Tai et al., 2015) as additional in- formation of the baseline ABS model. Conceptu- ally, the reason for using AMR for headline gen- eration is that information presented in AMR, such as predicate-argument structures and named entities, can be effective clues when producing shorter sum- maries (headlines) from original longer sentences. We expect that the quality of headlines will improve with this reasonable combination (ABS and AMR). 2 Attention-based summarization (ABS) ABS proposed in Rush et al. (2015) has achieved state-of-the-art performance on the benchmark data of headline generation including the DUC-2004 dataset (Over et al., 2007). Figure 1 illustrates the model structure of ABS. The model predicts a word sequence (summary) based on the combination of the neural network language model and an input sen- tence encoder. Let V be a vocabulary. xi is the i-th indicator vector corresponding to the i-th word in the input sentence. Suppose we have …

Content truncated to 500 words