An Empirical Study of the Influence of Argument Conciseness on Argument Effectiveness

An Empirical Study of the Influence of Argument Conciseness on Argument Effectiveness Giuseppe Carenini Intelligent Systems Program University of Pittsburgh, Pittsburgh, PA 15260, USA carenini@cs.pitt.edu Johanna D. Moore The Human Communication Research Centre, University of Edinburgh, 2 Buccleuch Place, Edinburgh EH8 9LW, UK. jmoore@cogsci.ed.ac.uk Abstract We have developed a system that generates evaluative arguments that are tailored to the user, properly arranged and concise. We have also developed an evaluation framework in which the effectiveness of evaluative arguments can be measured with real users. This paper presents the results of a formal experiment we have performed in our framework to verify the influence of argument conciseness on argument effectiveness 1 Introduction Empirical methods are critical to gauge the scalability and robustness of proposed approaches, to assess progress and to stimulate new research questions. In the field of natural language generation, empirical evaluation has only recently become a top research priority (Dale, Eugenio et al. 1998). Some empirical work has been done to evaluate models for generating descriptions of objects and processes from a knowledge base (Lester and Porter March 1997), text summaries of quantitative data (Robin and McKeown 1996), descriptions of plans (Young to appear) and concise causal arguments (McConachy, Korb et al. 1998). However, little attention has been paid to the evaluation of systems generating evaluative arguments, communicative acts that attempt to affect the addressee’s attitudes (i.e. evaluative tendencies typically phrased in terms of like and dislike or favor and disfavor). The ability to generate evaluative arguments is critical in an increasing number of online systems that serve as personal assistants, advisors, or shopping assistants1. For instance, a shopping assistant may need to compare two similar products and argue why its current user should like one more than the other. 1 See for instance www.activebuyersguide.com In the remainder of the paper, we first describe a computational framework for generating evaluative arguments at different levels of conciseness. Then, we present an evaluation framework in which the effectiveness of evaluative arguments can be measured with real users. Next, we describe the design of an experiment we ran within the framework to verify the influence of argument conciseness on argument effectiveness. We conclude with a discussion of the experiment’s results. 2 Generating concise evaluative arguments Often an argument cannot mention all the available evidence, usually for the sake of brevity. According to argumentation theory, the selection of what evidence to mention in an argument should be based on a measure of the evidence strength of support (or opposition) to the main claim of the argument (Mayberry and Golden 1996). Furthermore, argumentation theory suggests that for evaluative arguments the measure of evidence strength should be based on a model of the intended reader’s values and preferences. Following argumentation theory, we have designed an argumentative strategy for generating evaluative arguments that are properly arranged and concise (Carenini and Moore 2000). In our strategy, we assume that the reader’s values and preferences are represented as an additive multiattribute value function (AMVF), a conceptualization based on …

Content truncated to 500 words