Conditional Random Fields for Identifying Appropriate Types of Support for Propositions in Online User Comments

Proceedings of the 2nd Workshop on Argumentation Mining, pages 39–44, Denver, Colorado, June 4, 2015. c⃝2015 Association for Computational Linguistics Conditional Random Fields for Identifying Appropriate Types of Support for Propositions in Online User Comments Joonsuk Park Dept. of Computer Science Cornell University Ithaca, New York, USA jpark@cs.cornell.edu Arzoo Katiyar Dept. of Computer Science Cornell University Ithaca, New York, USA arzoo@cs.cornell.edu Bishan Yang Dept. of Computer Science Cornell University Ithaca, New York, USA bishan@cs.cornell.edu Abstract Park and Cardie (2014) proposed a novel task of automatically identifying appropriate types of support for propositions comprising online user comments, as an essential step toward au- tomated analysis of the adequacy of support- ing information. While multiclass Support Vector Machines (SVMs) proved to work rea- sonably well, they do not exploit the sequen- tial nature of the problem: For instance, veri- fiable experiential propositions tend to appear together, because a personal narrative typi- cally spans multiple propositions. Accord- ing to our experiments, however, Conditional Random Fields (CRFs) degrade the overall performance, and we discuss potential fixes to this problem. Nonetheless, we observe that the F1 score with respect to the unver- ifiable proposition class is increased. Also, semi-supervised CRFs with posterior regular- ization trained on 75% labeled training data can closely match the performance of a super- vised CRF trained on the same training data with the remaining 25% labeled as well. 1 Introduction The primary domain for argumentation mining has been professionally written text, such as parliamen- tary records, legal documents and news articles, which contain well-formed arguments consisting of explicitly stated premises and conclusions (Palau and Moens, 2009; Wyner et al., 2010; Feng and Hirst, 2011; Ashley and Walker, 2013). In con- trast, online user comments are often comprised of implicit arguments, which are conclusions with no explicitly stated premises1. For instance, in the fol- lowing user comment, neither of the two proposi- tions are supported with a reason or evidence. In other words, each of the two propositions is the con- clusion of its own argument, with no explicit support provided (thus called implicit arguments): All airfare costs should include the pas- senger’s right to check at least one stan- dard piece of baggage.A All fees should be fully disclosed at the time of airfare pur- chase, regardless of nature.B When the goal is to extract well-formed argu- ments from a given text, one may simply disregard such implicit arguments. (Villalba and Saint-Dizier, 2012; Cabrio and Villata, 2012). However, with the accumulation of a large amount of text consisting of implicit arguments, a means of assessing the ade- quacy of support in arguments has become increas- ingly desirable. It is not only beneficial for analyz- ing the strength of arguments, but also for helping commenters to construct better arguments by sug- gesting the appropriate types of support to be pro- vided. As an initial step toward automatically assess- ing the adequacy of support in arguments, Park and Cardie (2014) proposed a novel task of classifying each proposition based on the appropriate …

Content truncated to 500 words