Detecting and Characterizing Events

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1142–1152, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Detecting and Characterizing Events Allison J. B. Chaney Princeton University achaney@cs.princeton.edu Hanna Wallach Microsoft Research wallach@microsoft.com Matthew Connelly Columbia University mjc96@columbia.edu David M. Blei Columbia University david.blei@columbia.edu Abstract Significant events are characterized by interac- tions between entities (such as countries, or- ganizations, or individuals) that deviate from typical interaction patterns. Analysts, includ- ing historians, political scientists, and journal- ists, commonly read large quantities of text to construct an accurate picture of when and where an event happened, who was involved, and in what ways. In this paper, we present the Capsule model for analyzing documents to detect and characterize events of potential significance. Specifically, we develop a model based on topic modeling that distinguishes be- tween topics that describe “business as usual” and topics that deviate from these patterns. To demonstrate this model, we analyze a corpus of over two million U.S. State Department cables from the 1970s. We provide an open-source im- plementation of an inference algorithm for the model and a pipeline for exploring its results. 1 Introduction Foreign embassies of the United States government communicate with one another and with the U.S. State Department through diplomatic cables. The National Archive collects these cables in a corpus, which traces the (declassified) diplomatic history of the United States.1 The corpus contains, for example, over two million cables sent between 1973 and 1978. Most of these cables describe diplomatic “business as usual,” such as arrangements for visiting officials, 1 The National Archives’ corpus also includes messages sent by diplomatic pouch; however, for brevity, and at the risk of being imprecise, we also refer to these messages as “cables.” recovery of lost or stolen passports, or obtaining lists of names for meetings and conferences. For example, the embassies sent 8,635 cables during the week of April 21, 1975. Here is one, selected at random: Hoffman, UNESCO Secretariat, requested info from PermDel concerning an official in- vitation from the USG RE subject meeting scheduled 10–13 JUNE 1975, Madison, Wis- consin. Would appreciate info RE status of action to be taken in order to inform Secre- tariat. Hoffman communicating with Dr. John P. Klus RE list of persons to be invited. But, hidden in the corpus are also cables about im- portant diplomatic events—the cables and events that are most interesting to historians, political scientists, and journalists. For example, during that same week, the U.S. was in the last moments of the Vietnam War and, on April 30, 1975, lost its hold on Saigon. This marked the end of the war and induced a mass exodus of refugees. Here is one cable about this event: GOA program to move Vietnamese Refugees to Australia is making little progress and probably will not cover more than 100-200 persons. Press comment on smallness of program has recognized difficulty of getting Vietnamese out of Saigon, but “Canberra Times” Apr 25 sharply critical of …

Content truncated to 500 words