Ethics consideration sections in natural language processing papers

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4509 - 4516 December 7-11, 2022 ©2022 Association for Computational Linguistics Ethics consideration sections in natural language processing papers Luciana Benotti Universidad Nacional de Córdoba Via Libre, CONICET, Argentina luciana.benotti@unc.edu.ar Patrick Blackburn Philosophy and Science Studies IKH, Roskilde University, Denmark patrickb@ruc.dk Abstract In this paper we present the results of a manual classification of all ethical consideration sec- tions for ACL 2021. We also compare how many papers had an ethics consideration sec- tion per track and per world region in ACL 2021. We classified papers according to the ethical issues covered (research benefits, po- tential harms, and vulnerable groups affected) and whether the paper was marked as requiring ethics review by at least one reviewer. More- over, we discuss recurring obstacles we have observed (highlighting some interesting texts we found along the way) and conclude with three suggestions. We think that this paper may be useful for anyone who needs to write — or review — an ethics section and would like to get an overview of what others have done. 1 Introduction The first conference of the Association for Com- putational Linguistics (ACL) to include an ethics advisory committee was the one organized by its North American Chapter (NAACL) in 2021. Since then all ACL conferences have had one, including all chapter conferences, and the largest conferences ACL-IJCNLP and EMNLP. The ACL 2021 web- page call for papers says the following.1 Authors will be allowed extra space after the 8th page for a broader impact statement or other discussion of ethics. Note that though the ethical consideration paragraph is not manda- tory, authors of papers working with sensitive data or on sensitive tasks that do not sufficiently discuss these issues may receive a conditional acceptance recommendation. This paper examines the current state of the NLP research community’s discussion of the ethical im- pact of its work, as reflected in its publications. We first pose five empirical questions, and then explore 1Source https://2021.aclweb.org/calls/papers/ possible obstacles to the discussion. The paper is organized as follows. Section 2 briefly reviews re- lated work, giving references relevant to several areas of NLP. In Section 3 we explore the follow- ing five questions regarding ethical consideration sections in ACL 2021 papers: • What percentage of papers include an ethical considerations section (ECS)? • Are there some tracks that stand out, either positively or negatively? • What types of ethical questions are addressed in ECSs, and in what proportion? • Are the papers with ECSs that went through ethics review different from those that did not? • Are there differences between countries with respect to the ECSs? In Section 4 we consider obstacles that may im- pede the discussion of ethical issues in research papers, and how they might be overcome. In Sec- tion 5 we propose three concrete suggestions that can be useful in the future for authors, reviewers and ethics chairs. After the conclusions in Sec- tion 6 we …

Content truncated to 500 words