Joint Optimization of User-desired Content in Multi-document Summaries by Learning from User Feedback

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1353–1363 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1124 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1353–1363 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1124 Joint Optimization of User-desired Content in Multi-document Summaries by Learning from User Feedback Avinesh P.V.S and Christian M. Meyer Research Training Group AIPHES and UKP Lab Computer Science Department, Technische Universit¨at Darmstadt www.aiphes.tu-darmstadt.de, www.ukp.tu-darmstadt.de Abstract In this paper, we propose an extractive multi-document summarization (MDS) system using joint optimization and active learning for content selection grounded in user feedback. Our method interactively obtains user feedback to gradually im- prove the results of a state-of-the-art inte- ger linear programming (ILP) framework for MDS. Our methods complement fully automatic methods in producing high- quality summaries with a minimum num- ber of iterations and feedbacks. We conduct multiple simulation-based exper- iments and analyze the effect of feedback- based concept selection in the ILP setup in order to maximize the user-desired content in the summary. 1 Introduction The task of producing summaries from a clus- ter of multiple topic-related documents has gained much attention during the Document Understand- ing Conference1 (DUC) and the Text Analysis Conference2 (TAC) series. Despite a lot of re- search in this area, it is still a major challenge to automatically produce summaries that are on par with human-written ones. To a large extent, this is due to the complexity of the task: a good sum- mary must include the most relevant information, omit redundancy and irrelevant information, sat- isfy a length constraint, and be cohesive and gram- matical. But an even bigger challenge is the high degree of subjectivity in content selection, as it can be seen in the small overlap of what is considered 1http://duc.nist.gov/ 2http://www.nist.gov/tac/ important by different users. Optimizing a sys- tem towards one single best summary that fits all users, as it is assumed by current state-of-the-art systems, is highly impractical and diminishes the usefulness of a system for real-world use cases. In this paper, we propose an interactive concept- based model to assist users in creating a personal- ized summary based on their feedback. Our model employs integer linear programming (ILP) to max- imize user-desired content selection while using a minimum amount of user feedback and iterations. In addition to the joint optimization framework us- ing ILP, we explore pool-based active learning to further reduce the required feedback. Although there have been previous attempts to assist users in single-document summarization, no existing work tackles the problem of multi-document summaries using optimization techniques for user feedback. Additionally, most existing systems produce only a single, globally optimal solution. Instead, we put the human in the loop and create a personal- ized summary that better captures the users’ needs and their different notions of importance. Need for personalization. Table 1 shows the ROUGE scores (Lin, 2004) of multiple existing summarization systems, …

Content truncated to 500 words