Exploring Differential Topic Models for Comparative Summarization of Scientific Papers

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1028–1038, Osaka, Japan, December 11-17 2016. Exploring Differential Topic Models for Comparative Summarization of Scientific Papers Lei He*#, Wei Li*, Hai Zhuge*# # System Analytics Research Institute, Aston University, Birmingham, UK * Key Lab of Intelligent Information Processing, ICT, CAS， University of Chinese Academy of Sciences, Beijing, China hel2@aston.ac.uk,weili.ict.kg@gmail.com,zhuge@ict.ac.cn Abstract This paper investigates differential topic models (dTM) for summarizing the differences among docu- ment groups. Starting from a simple probabilistic generative model, we propose dTM-SAGE that explic- itly models the deviations on group-specific word distributions to indicate how words are used differen- tially across different document groups from a background word distribution. It is more effective to cap- ture unique characteristics for comparing document groups. To generate dTM-based comparative sum- maries, we propose two sentence scoring methods for measuring the sentence discriminative capacity. Experimental results on scientific papers dataset show that our dTM-based comparative summarization methods significantly outperform the generic baselines and the state-of-the-art comparative summariza- tion methods under ROUGE metrics. 1 Introduction Today, the interconnected nature of real-world applications brings more cross-field research problems leading to a much closer relationship between research areas. Real-world challenges require research- ers to quickly get acquainted with knowledge in other areas. For example, imagine a researcher who is familiar with topic models wants to extend her research to opinion summarization. She would be more interested in finding out the current development of sentiment analysis and how topic models can be used in sentiment analysis, rather than the common background knowledge such as topic models and basic NLP technologies. Such a real-world demand encourages the study of multi-document compara- tive summarization for scientific papers in multiple subject areas. This paper presents the initial study on this problem. Comparative summarization aims at summarizing the differences among document groups (Wang et al., 2012). The core is to compare different topics and find unique characteristics for each document group. The main motivation of this paper is to apply dTM to comparative summarization and to model the group-specific topics to capture the unique word usage for characterising documents in the same group. To our best knowledge, there is no previous study providing in-depth model analysis and de- tailed experimental results on dTM applied for comparative summarization. We first propose a probabilistic generative model dTM-Dirichlet to model the group-specific word distributions to capture the unique word usage for each document group. However, dTM-Dirichlet is not a truly differential topic model and it suffers from the problems of high inference cost, over- parameterization and lack of sparsity. Evolving from the idea of SAGE (Eisenstein et al., 2011), we develop dTM-SAGE to make the word probability distributions for each document group to share a common background word distribution and explicitly models how words are used differently in each group from the background word distribution. Our main contributions include the following two points: (1) we propose dTM to capture unique characteristics of each document group in the application …

Content truncated to 500 words