A Corpus of eRulemaking User Comments for Measuring Evaluability of Arguments

A Corpus of eRulemaking User Comments for Measuring Evaluability of Arguments Joonsuk Park and Claire Cardie Department of Computer Science, Williams College, Massachusetts, USA Department of Computer Science, Cornell University, New York, USA jpark@cs.williams.edu, cardie@cs.cornell.edu Abstract eRulemaking is a means for government agencies to directly reach citizens to solicit their opinions and experiences regarding newly proposed rules. The effort, however, is partly hampered by citizens’ comments that lack reasoning and evidence, which are largely ignored since government agencies are unable to evaluate the validity and strength. We present Cornell eRulemaking Corpus – CDCP, an argument mining corpus annotated with argumentative structure information capturing the evaluability of arguments. The corpus consists of 731 user comments on Consumer Debt Collection Practices (CDCP) rule by the Consumer Financial Protection Bureau (CFPB); the resulting dataset contains 4931 elementary unit and 1221 support relation annotations. It is a resource for building argument mining systems that can not only extract arguments from unstructured text, but also identify what additional information is necessary for readers to understand and evaluate a given argument. Immediate applications include providing real-time feedback to commenters, specifying which types of support for which propositions can be added to construct better-formed arguments. Keywords: argument mining, e-government, e-rulemaking, text analytics 1 Introduction The U.S. federal agencies amend rules in a highly trans- parent manner, inviting public participation as they are finalized. This is legally ensured in part by the require- ment that agencies publish descriptions and rationale behind newly proposed rules and solicit feedback from the public (Park et al., 2012; Farina and Newhart, 2013). However, the public participation tends to be dominated by large corporations and interest groups (CSFFR, 2009); eRulemaking is an ongoing effort to promote citizens’ participation in federal policymaking by using the latest information technology to directly reach citizens and incorporate their feedback (Lubbers et al., 2012). Government agencies consider reasoning and validity of supporting evidence, rather than a mere number of citizens supporting an argument, to determine how the rules should be adjusted to meet the needs of those who are directly affected. Thus, useful feedback consists of clear reasoning and objective evidence supporting factual claims (Park et al., 2015). However, many comments are not written this way, thwarting the government agencies’ effort to communicate with citizens. Consider the following comments from www. regulationroom.org, an eRulemaking website: (1) $400 is enough compensation,A as it can cover a one-way fare across the US.B I checked in a passenger on a $98.00 fare from east coast to Las Vegas the other day.C (2) All airfare costs should include the passen- ger’s right to check at least one standard piece of baggage.A All fees should be fully disclosed at the time of airfare purchase, regardless of nature (i.e. optional or mandatory).B Any changes in fees should be identified by air carriers at least 6 months prior to taking effect.C Comment 1 consists of propositions in support relations that collectively form a single argument: Proposition 1.C is an anecdotal evidence supporting Proposition 1.B, which …

Content truncated to 500 words