AttSum: Joint Learning of Focusing and Summarization with Neural Attention

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 547–556, Osaka, Japan, December 11-17 2016. AttSum: Joint Learning of Focusing and Summarization with Neural Attention Ziqiang Cao1 Wenjie Li1 Sujian Li2 Furu Wei3 Yanran Li1 1Department of Computing, The Hong Kong Polytechnic University, Hong Kong 2Key Laboratory of Computational Linguistics, Peking University, MOE, China 3Microsoft Research, Beijing, China {cszqcao, cswjli, csyli}@comp.polyu.edu.hk lisujian@pku.edu.cn furu@microsoft.com Abstract Query relevance ranking and sentence saliency ranking are the two main tasks in extractive query-focused summarization. Previous supervised summarization systems often perform the two tasks in isolation. However, since reference summaries are the trade-off between relevance and saliency, using them as supervision, neither of the two rankers could be trained well. This pa- per proposes a novel summarization system called AttSum, which tackles the two tasks jointly. It automatically learns distributed representations for sentences as well as the document clus- ter. Meanwhile, it applies the attention mechanism to simulate the attentive reading of human behavior when a query is given. Extensive experiments are conducted on DUC query-focused summarization benchmark datasets. Without using any hand-crafted features, AttSum achieves competitive performance. We also observe that the sentences recognized to focus on the query indeed meet the query need. 1 Introduction Query-focused summarization (Dang, 2005) aims to create a brief, well-organized and fluent summary that answers the need of the query. It is useful in many scenarios like news services and search engines, etc. Nowadays, most summarization systems are under the extractive framework which directly selects existing sentences to form the summary. Basically, there are two major tasks in extractive query-focused summarization, i.e., to measure the saliency of a sentence and its relevance to a user’s query. After a long period of research, learning-based models like Logistic Regression (Li et al., 2013) etc. have become growingly popular in this area. However, most current supervised summarization systems often perform the two tasks in isolation. Usually, they design query-dependent features (e.g., query word overlap) to learn the relevance ranking, and query-independent features (e.g., term frequency) to learn the saliency ranking. Then, the two types of features are combined to train an overall ranking model. Note that the only supervision available is the reference summaries. Humans write summaries with the trade-off between relevance and saliency. Some salient content may not appear in reference summaries if it fails to respond to the query. Likewise, the content relevant to the query but not representative of documents will be excluded either. As a result, in an isolated model, weights for neither query-dependent nor query-independent features could be learned well from reference summaries. In addition, when measuring the query relevance, most summarization systems merely make use of surface features like the TF-IDF cosine similarity between a sentence and the query (Wan and Xiao, 2009). However, relevance is not similarity. Take the document cluster “d360f” in DUC1 2005 as an ex- ample. It has the following query: What are the benefits of drug legalization? Here, “Drug legalization” are the key words with …

Content truncated to 500 words