Are You Convinced? Choosing the More Convincing Evidence with a Siamese Network

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 967–976 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 967 Are You Convinced? Choosing the More Convincing Evidence with a Siamese Network Martin Gleize∗, Eyal Shnarch∗, Leshem Choshen, Lena Dankin, Guy Moshkowich, Ranit Aharonov, Noam Slonim IBM Research martin.gleize@ie.ibm.com, guy.moshkowich@ibm.com, {eyals, leshem.choshen, lenad, ranita, noams}@il.ibm.com Abstract Machines capable of responding and interact- ing with humans in helpful ways have become ubiquitous. We now expect them to discuss with us the more delicate questions in our world, and they should do so armed with effec- tive arguments. But what makes an argument more persuasive? What will convince you? In this paper, we present a new data set, IBM- EviConv, of pairs of evidence labeled for con- vincingness, designed to be more challenging than existing alternatives. We also propose a Siamese neural network architecture shown to outperform several baselines on both a prior convincingness data set and our own. Finally, we provide insights into our experimental re- sults and the various kinds of argumentative value our method is capable of detecting. 1 Introduction The most interesting questions in life do not have a simple factual answer. Rather, they have pros and cons associated with them. When opposing sides debate such questions, each side aims to present the most convincing arguments for its point of view, typically by raising various claims and sup- porting them with relevant pieces of evidence. Ide- ally, the arguments by both sides are then carefully compared, as part of the decision process. Automatic methods for this process of argu- mentation and debating are developed within the field of Computational Argumentation. which fo- cuses on methods for argument detection (Lippi and Torroni, 2016; Levy et al., 2014; Rinott et al., 2015) and revealing argument relations (Stab and Gurevych, 2014, 2017). Recently, IBM introduced Project Debater, the first AI system able to debate humans on complex topics. Project Debater participated in a live de- bate against a world champion debater, and was ∗First two authors contributed equally. able to mine arguments and use them to compose a speech supporting its side of the debate. In ad- dition, it was able to rebut its human competitor.1 The technology that underlies such a system is in- tended to enhance decision making. In this work we target the task of assessing ar- gument convincingness, and more specifically, we focus on evidence convincingness – given texts representing evidence for a given debatable topic, identify the more convincing ones. Theoretical works have analyzed the factors that make an argument more convincing (e.g., Boudry et al., 2015). This work is an empirical one in the line of (Persing and Ng, 2017; Tan et al., 2016). To the best of our knowledge, this is the first work on evidence convincingness. Most similar to our work is that of Habernal and Gurevych (2016a) who are the first to directly compare pairs of arguments (previous works com- pared documents). They released …

Content truncated to 500 words