HITIQA: Towards Analytical Question Answering

HITIQA: Towards Analytical Question Answering Sharon Small1, Tomek Strzalkowski1, Ting Liu1, Sean Ryan1, Robert Salkin1, Nobuyuki Shimizu1, Paul Kantor2, Diane Kelly2, Robert Rittman2, Nina Wacholder2 1The State University of New York at Albany 1400 Washington Avenue Albany, NY 12222 {small,tomek,tl7612,seanryan, rs6021,ns3202}@albany.edu 2Rutgers University 4 Huntington Street New Brunswick, NJ 08904 {kantor,diane,hitiqa, wacholder}@scils.rutgers.edu Abstract In this paper we describe the analytic question answering system HITIQA (High- Quality Interactive Question Answering) which has been developed over the last 2 years as an advanced research tool for information analysts. HITIQA is an interactive open- domain question answering technology designed to allow analysts to pose complex exploratory questions in natural language and obtain relevant information units to prepare their briefing reports. The system uses novel data-driven semantics to conduct clarification dialogue with the user that a explores the scope and the context of the desired answer space. The system has a undergone extensive hands-on evaluations by group of intelligence analysts. This evaluation validated the overall approach in HITIQA but also exposed limitations of the current prototype. 1 Introduction Our objective in HITIQA is to allow the user to submit exploratory, analytical questions, such as “What has been Russia’s reaction to the U.S. bombing of Kosovo?” The distinguishing property of such questions is that one cannot generally anticipate what might constitute the answer. While certain types of things may be expected (e.g., diplomatic statements), the answer is heavily conditioned by what information is in fact available on the topic. From a practical viewpoint, analytical questions are often underspecified, thus casting a broad net on a space of possible answers. Questions posed by professional analysts are aimed to probe the available data along certain dimensions. The results of these probes determine follow up questions, if necessary. Furthermore, at any stage clarifications may be needed to adjust the scope and intent of each question. Figure 1 shows a fragment of an analytical session with HITIQA; note that these questions are not aimed at factoids, despite their simple form. User: What is the history of the nuclear arms program linking Iraq and other countries in the region? HITIQA: [responses and clarifications] User: Who financed the nuclear arms program in Iraq? HITIQA:… User: Has Iraq been able to import uranium? HITIQA:… User: What type of debt does exist between Iraq and her trading partners in the region? FIGURE 1: A fragment of an analyst’s session with HITIQA HITIQA project is part of the ARDA AQUAINT program that aims to make significant advances in the state of the art of automated question answering. In this paper we focus on three aspects of our work: 1. Question Semantics: how the system “understands” user requests 2. Human-Computer Dialogue: how the user and the system negotiate this understanding 3. User Evaluations and Results 2 Factoid vs. Analytical QA There are significant differences between factoid, or fact-finding, and analytical question answering. A factoid question is normally understood to seek a piece of information that would make a corresponding statement true (i.e., it becomes a …

Content truncated to 500 words