Using External Resources and Joint Learning for Bigram Weighting in ILP-Based Multi-Document Summarization

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 778–787, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Using External Resources and Joint Learning for Bigram Weighting in ILP-Based Multi-Document Summarization Chen Li1, Yang Liu1, Lin Zhao2 1 Computer Science Department, The University of Texas at Dallas Richardson, Texas 75080, USA 2 Research and Technology Center, Robert Bosch LLC Palo Alto, California 94304, USA {chenli,yangl@hlt.utdallas.edu} {lin.zhao@us.bosch.com} Abstract Some state-of-the-art summarization systems use integer linear programming (ILP) based methods that aim to maximize the important concepts covered in the summary. These con- cepts are often obtained by selecting bigrams from the documents. In this paper, we improve such bigram based ILP summarization meth- ods from different aspects. First we use syn- tactic information to select more important bi- grams. Second, to estimate the importance of the bigrams, in addition to the internal features based on the test documents (e.g., document frequency, bigram positions), we propose to extract features by leveraging multiple exter- nal resources (such as word embedding from additional corpus, Wikipedia, Dbpedia, Word- Net, SentiWordNet). The bigram weights are then trained discriminatively in a joint learn- ing model that predicts the bigram weights and selects the summary sentences in the ILP framework at the same time. We demonstrate that our system consistently outperforms the prior ILP method on different TAC data sets, and performs competitively compared to other previously reported best results. We also con- ducted various analyses to show the contribu- tions of different components. 1 Introduction Extractive summarization is a sentence selection problem: identifying important summary sentences from one or multiple documents. Many methods have been developed for this problem, including su- pervised approaches that use a classifier to predict whether or not a sentence is in the summary, or un- supervised methods such as graph-based approaches to rank the sentences. Recently global optimiza- tion methods such as integer linear programming (ILP) have been shown to be quite powerful for this task. For example, Gillick et al. (2009) used ILP to achieve the best result in the TAC 09 summa- rization task. The core idea of this summarization method is to select the summary sentences by maxi- mizing the sum of the weights of the language con- cepts that appear in the summary. Bigrams are often used as these language concepts because Gillick et al. (2009) stated that the bigrams gave consistently better performance than unigrams or trigrams for a variety of ROUGE measures. The association be- tween the language concepts and sentences serves as the constraints. This ILP method is formally rep- resented as below (see (Gillick et al., 2009) for more details): max � i wici (1) s.t. sjOccij ≤ ci (2) � j sjOccij ≥ ci (3) � j ljsj ≤ L (4) ci ∈ {0, 1} ∀i (5) sj ∈ {0, 1} ∀j (6) ci and sj are binary variables that indicate the pres- ence of a concept and a sentence respectively. …

Content truncated to 500 words