Show Me Your Evidence - an Automatic Method for Context Dependent Evidence Detection

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 440–450, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Show Me Your Evidence – an Automatic Method for Context Dependent Evidence Detection Ruty Rinott1, Lena Dankin1, Carlos Alzate2, Mitesh M. Khapra3, Ehud Aharoni1, Noam Slonim1 1IBM Research - Haifa, Mount Carmel, Haifa, 31905, Israel, 2IBM Research - Ireland, Damastown Industrial Estate, Dublin 15, Ireland, 3IBM Research - Bangalore, India, {rutyr,lenad,aehud,noams}@il.ibm.com carlos.alzate@ie.ibm.com mikhapra@in.ibm.com Abstract Engaging in a debate with oneself or others to take decisions is an integral part of our day-to- day life. A debate on a topic (say, use of per- formance enhancing drugs) typically proceeds by one party making an assertion/claim (say, PEDs are bad for health) and then providing an evidence to support the claim (say, a 2006 study shows that PEDs have psychiatric side effects). In this work, we propose the task of automatically detecting such evidences from unstructured text that support a given claim. This task has many practical applications in decision support and persuasion enhancement in a wide range of domains. We first introduce an extensive benchmark data set tailored for this task, which allows training statistical mod- els and assessing their performance. Then, we suggest a system architecture based on super- vised learning to address the evidence detec- tion task. Finally, promising experimental re- sults are reported. 1 Introduction In recent years there has been a growing interest in the area of argumentation mining (Green et al., 2014; Cardie et al., 2015; Wells, 2014). Part of this awak- ening is the The DebaterTM project1 whose goal is to develop technologies that will assist humans to de- bate and reason, e.g., by automatically suggesting argu- ments relevant to an examined topic. The minimal def- inition of such an argument (Walton, 2009) is a set of statements, made up of three parts – a claim (aka con- clusion, proposition), a set of evidence (aka premises), and an inference from the evidence to the claim. Need- less to say, evidence plays a critical role in a persuasive argument. In most debate related skills, such as natural lan- guage understanding and generation, humans currently have an inherent advantage over a machine. However, in the ability to provide high quality and diverse evi- dence, machines have a very promising potential, being 1http://researcher.ibm.com/researcher/ view_group.php?id=5443 able to swiftly process large quantities of information. Nonetheless, since most of the relevant information is represented by unstructured text, successfully exploit- ing these resources requires the ability to identify evi- dence in free text. This is exactly the focus of our work. Specifically, we formally define the task of evidence detection, introduce an architecture for attacking this problem, and demonstrate its performance over dedi- cated manually labeled data. Before defining the task formally, we introduce three concepts which will be used throughout this paper. These concepts were earlier defined in (Aharoni et al., 2014) and we use the same definitions here. Topic: a short phrase that frames the discussion. …

Content truncated to 500 words