Give Me Convenience and Give Her Death: Who Should Decide What Uses of NLP are Appropriate, and on What Basis?

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2908–2913 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 2908 Give Me Convenience and Give Her Death: Who Should Decide What Uses of NLP are Appropriate, and on What Basis? Kobi Leins Jey Han Lau Timothy Baldwin School of Computing and Information Systems, The University of Melbourne {kleins,laujh,tbaldwin}@unimelb.edu.au Abstract As part of growing NLP capabilities, coupled with an awareness of the ethical dimensions of research, questions have been raised about whether particular datasets and tasks should be deemed off-limits for NLP research. We ex- amine this question with respect to a paper on automatic legal sentencing from EMNLP 2019 which was a source of some debate, in asking whether the paper should have been allowed to be published, who should have been charged with making such a decision, and on what ba- sis. We focus in particular on the role of data statements in ethically assessing research, but also discuss the topic of dual use, and examine the outcomes of similar debates in other scien- tific disciplines. 1 Introduction NLP tools are increasingly being deployed in the wild with potentially profound societal implica- tions. Alongside the rise in technical capabilities has been a growing awareness of the moral obli- gation of the field to self-assess issues including: dataset and system bias (Zhao et al., 2017), dataset ethics (Bender and Friedman, 2018), and dual use (Hovy and Spruit, 2016). More recently, there has also been vigorous debate on whether it is ethical for the community to work on certain topics or data types. This paper aims to investigate this issue, focused around the examination of a paper recently published at EMNLP 2019 on automatic prison term prediction by Chen et al. (2019). Specifi- cally, the paper in question proposes a neural model which performs structured prediction of the indi- vidual charges laid against an individual, and the prison term associated with each, which can pro- vide an overall prediction of the prison term asso- ciated with the case. This model was constructed using a large-scale dataset of real-world Chinese court cases. The primary question we attempt to address in this paper is on what basis a given paper satisfies basic ethical requirements for publication, in ad- dition to examining the related question of who should make this judgement. Note that our intention is in no way to victimise the authors of the paper in question, but rather to use it as a test case to objectively ground an ethical assessment. The authors did highlight potential ethical concerns of its application, but missed the point that there are data ethics issue in the first place. Note also that, given the topic of the pa- per, we will focus somewhat on NLP applications in the legal domain, but the majority of the find- ings/recommendations generalise and will be of equal relevance to other domains. 2 Case Study in Ethical NLP Publication 2.1 Data ethics The first dimension to …

Content truncated to 500 words