IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Tasks

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2277 - 2287 May 22-27, 2022 c⃝2022 Association for Computational Linguistics IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Tasks Liying Cheng∗1,2 Lidong Bing†1 Ruidan He1 Qian Yu‡3 Yan Zhang‡4 Luo Si1 1DAMO Academy, Alibaba Group 2Singapore University of Technology and Design 3The Chinese University of Hong Kong 4 National University of Singapore {liying.cheng, l.bing, ruidan.he, luo.si}@alibaba-inc.com yuqian@se.cuhk.edu.hk eleyanz@nus.edu.sg Abstract Traditionally, a debate usually requires a man- ual preparation process, including reading plenty of articles, selecting the claims, iden- tifying the stances of the claims, seeking the evidence for the claims, etc. As the AI debate attracts more attention these years, it is worth exploring the methods to automate the tedious process involved in the debating system. In this work, we introduce a comprehensive and large dataset named IAM, which can be ap- plied to a series of argument mining tasks, in- cluding claim extraction, stance classification, evidence extraction, etc. Our dataset is col- lected from over 1k articles related to 123 top- ics. Near 70k sentences in the dataset are fully annotated based on their argument properties (e.g., claims, stances, evidence, etc.). We fur- ther propose two new integrated argument min- ing tasks associated with the debate prepara- tion process: (1) claim extraction with stance classification (CESC) and (2) claim-evidence pair extraction (CEPE). We adopt a pipeline approach and an end-to-end method for each integrated task separately. Promising experi- mental results are reported to show the values and challenges of our proposed tasks, and mo- tivate future research on argument mining. 1 1 Introduction Debating has a long history and wide application scenarios in education field (Stab and Gurevych, 2014; Persing and Ng, 2016; Stab and Gurevych, 2017), political domain (Lippi and Torroni, 2016; Duthie et al., 2016; Menini et al., 2018), legal ac- tions (Mochales and Moens, 2011; Grabmair et al., 2015; Teruel et al., 2018), etc. It usually involves tons of manual preparation steps, including read- ing the articles, selecting the claims, identifying ∗Liying Cheng is under the Joint Ph.D. Program between Alibaba and Singapore University of Technology and Design. † Corresponding author. ‡Qian Yu and Yan Zhang were interns at Alibaba. 1Our code, data and leaderboard are available at https: //github.com/LiyingCheng95/IAM. Debating topic Multiple articles Claim 1 Evidence for Claim 1 Claim n Evidence for Claim n ... Stance Stance Figure 1: A flow chart showing the debating prepara- tion process. the claim stances to the topics, looking for the evidence of the claims, etc. Since the machine has shown promising potential in processing large quantities of information in many other natural lan- guage processing tasks, it is also worthwhile to explore the methods for automating the manual process involved in debating. Argument mining (AM), as the core of a debat- ing system (Bar-Haim et al., 2021), has received more attention in the past few years. Several AM tasks and datasets have been proposed to work to- …

Content truncated to 500 words