A Framework for Comparing Groups of Documents

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 840–845, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. A Framework for Comparing Groups of Documents Arun S. Maiya Institute for Defense Analyses — Alexandria, VA, USA amaiya@ida.org Abstract We present a general framework for com- paring multiple groups of documents. A bipartite graph model is proposed where document groups are represented as one node set and the comparison criteria are represented as the other node set. Using this model, we present basic algorithms to extract insights into similarities and differ- ences among the document groups. Fi- nally, we demonstrate the versatility of our framework through an analysis of NSF funding programs for basic research. 1 Introduction and Motivation Given multiple sets (or groups) of documents, it is often necessary to compare the groups to identify similarities and differences along different dimen- sions. In this work, we present a general frame- work to perform such comparisons for extraction of important insights. Indeed, many real-world tasks can be framed as a problem of comparing two or more groups of documents. Here, we pro- vide two motivating examples. 1. Program Reviews. To better direct research efforts, funding organizations such as the National Science Foundation (NSF), the National Institutes of Health (NIH), and the Department of Defense (DoD), are often in the position of reviewing re- search programs via their artifacts (e.g., grant ab- stracts, published papers, and other research de- scriptions). Such reviews might involve identify- ing overlaps across different programs, which may indicate a duplication of effort. It may also involve the identification of unique, emerging, or dimin- ishing topics. A “document group” here could be defined either as a particular research program that funds many organizations, the totality of funded research conducted by a specific organization, or all research associated with a particular time pe- riod (e.g., fiscal year). In all cases, the objective is to draw comparisons between groups by compar- ing the document sets associated with them. 2. Intelligence. In the areas of defense and in- telligence, document sets are sometimes obtained from different sources or entities. For instance, the U.S. Armed Forces sometimes seize documents during raids of terrorist strongholds.1 Similarities between two document sets (each captured from a different source) can potentially be used to infer a non-obvious association between the sources. Of course, there are numerous additional examples across many domains (e.g., comparing different news sources, comparing the reviews for several products, etc.). Given the abundance of real-world applications as illustrated above, it is surprising, then, that there are no existing general-purpose ap- proaches for drawing such comparisons. While there is some previous work on the comparison of document sets (referred to as comparative text mining), these existing approaches lack the gener- ality to be widely applicable across different use case scenarios with different comparison criteria. Moreover, much of the work in the area focuses largely on the summarization of shared or un- shared topics among document groups (e.g., Wan …

Content truncated to 500 words