APE: Argument Pair Extraction from Peer Review and Rebuttal via Multi-task Learning

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 7000–7011, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 7000 APE: Argument Pair Extraction from Peer Review and Rebuttal via Multi-task Learning Liying Cheng∗1,2 Lidong Bing2 Qian Yu†3 Wei Lu1 Luo Si2 1Singapore University of Technology and Design 2DAMO Academy, Alibaba Group 3The Chinese University of Hong Kong {liying.cheng, l.bing, luo.si}@alibaba-inc.com yuqian@se.cuhk.edu.hk luwei@sutd.edu.sg Abstract Peer review and rebuttal, with rich interactions and argumentative discussions in between, are naturally a good resource to mine arguments. However, few works study both of them si- multaneously. In this paper, we introduce a new argument pair extraction (APE) task on peer review and rebuttal in order to study the contents, the structure and the connec- tions between them. We prepare a challeng- ing dataset that contains 4,764 fully annotated review-rebuttal passage pairs from an open re- view platform to facilitate the study of this task. To automatically detect argumentative propositions and extract argument pairs from this corpus, we cast it as the combination of a sequence labeling task and a text relation classification task. Thus, we propose a multi- task learning framework based on hierarchical LSTM networks. Extensive experiments and analysis demonstrate the effectiveness of our multi-task framework, and also show the chal- lenges of the new task as well as motivate fu- ture research directions. 1 1 Introduction Argument mining is an important research field that attracts growing attention in recent years (Lawrence and Reed, 2019). It is applied in real-world applications such as legal documents (Mochales and Moens, 2011; Poudyal, 2015), on- line debates (Boltuˇzi´c and ˇSnajder, 2015; Abbott et al., 2016), persuasive essays (Stab and Gurevych, 2014; Persing and Ng, 2016), etc. Most works in ar- gument mining field focus on modeling arguments in monologues. Those focusing on the detection of agreement and disagreement in online interactions ∗Liying Cheng is under the Joint Ph.D. Program between Alibaba and Singapore University of Technology and Design. †Qian Yu was an intern at Alibaba. 1Our code and data are available at https://github. com/LiyingCheng95/ArgumentPairExtraction. Review: [The authors introduce an extension of Continuous Ranked Probability Scores (CRPS) to the time-to-event setting termed Survival-CRPS for both right censored and interval- censored event data.]NON-ARGU ... [The claim that the pro- posed approach constitutes the first time a scoring rule other than maximum likelihood seems too strong, unnecessary and irrelevant to the value of the presented work.]REVIEW-1 [It is not clear how did the authors handle the irregularity (in time) of EHR encounters in the context of an RNN specification. Also, if the RNN specification considered is similar to Martinsson, 2016, why this wasn’t considered as a competing model in the experiments?]REVIEW-2 ... Rebuttal: [Dear AnonReviewer3, we thank you for the time you spent reviewing and for the thoughtful comments.]NON-ARGU [You point out that our claim that the approach constitutes the first time a scoring rule other than maximum likelihood has been used seems too strong. ... However we have recently come across works such as …

Content truncated to 500 words