“Who was Pietro Badoglio?” Towards a QA system for Italian History

“Who was Pietro Badoglio?” Towards a QA system for Italian History Stefano Menini1-2, Rachele Sprugnoli1-2, Antonio Uva2 1Fondazione Bruno Kessler, 2Universit`a di Trento 1Via Sommarive 18, 38123 Povo (TN), Italy 2Via Sommarive 9, 38123 Povo (TN), Italy {menini;sprugnoli}@fbk.eu; antonio.uva@unitn.it Abstract This paper presents QUANDHO (QUestion ANswering Data for italian HistOry), an Italian question answering dataset created to cover a specific domain, i.e. the history of Italy in the first half of the XX century. The dataset includes questions manually classified and annotated with Lexical Answer Types, and a set of question-answer pairs. This resource, freely available for research purposes, has been used to retrain a domain independent question answering system so to improve its performances in the domain of interest. Ongoing experiments on the development of a question classifier and an automatic tagger of Lexical Answer Types are also presented. Keywords: question answering, dataset creation, annotation 1. Introduction Question Answering (QA) systems provide users, who ask questions in natural language, with a short passage contain- ing the answer and some context to validate it (Hirschman and Gaizauskas, 2001). The development of such sys- tems requires a multidisciplinary approach combining tech- niques of different fields: it is not by chance that the lit- erature reports various studies dealing with QA systems from different perspectives, e.g. Information Retrieval (Kolomiyets and Moens, 2011) and Semantic Web (Lopez et al., 2011). In the last few years, QA has received a lot of atten- tion thanks to the success of the IBM’s Watson system in the Jeopardy! game (Ferrucci et al., 2010) and to its application to the clinical domain (Ferrucci et al., 2013). In addition, the organization of international evaluation campaigns such as TREC in the USA (Voorhees et al., 2005)1, CLEF in Europe (Mothe et al., 2015)2 and NT- CIR in Japan3 fostered the extension of QA research to languages other than English. Nevertheless only two sys- tems, both domain independent, are currently available for Italian, namely Wikiedi4 and the Italian Minimal Structural Reranking Pipeline (henceforth, It-MSRP) (Uva and Mos- chitti, 2015). In this paper we describe QUANDHO (QUestion ANswer- ing Data for italian HistOry), an Italian question answer- ing dataset that includes questions manually classified and annotated with lexical answer types, and a set of question- answer pairs. This dataset, freely available to the research community5, focuses on a specific domain, i.e. the Italian history of the first half of the XX century. In addition, we present the evaluation of Wikiedi and It-MSRP on this do- main and we report on the retraining of the latter using our dataset so to improve its precision and make it an effective 1http://trec.nist.gov/ 2http://www.clef-initiative.eu/ 3http://research.nii.ac.jp/ntcir/ index-en.html 4http://www.wikiedi.it/ 5https://dh.fbk.eu/technologies/quandho tool to be incorporated in a real application. The final aim is to integrate an interactive version of the QA system in ALCIDE (Moretti et al., 2014), a web-based platform for historical content analysis. ALCIDE contains different cor- pora related to the historical domain among which the com- plete corpus of Alcide De Gasperi’s6 writings is the …

Content truncated to 500 words