How Good Is NLP? A Sober Look at NLP Tasks through the Lens of Social Impact

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3099–3113 August 1–6, 2021. ©2021 Association for Computational Linguistics 3099 How Good Is NLP? A Sober Look at NLP Tasks through the Lens of Social Impact Zhijing Jin Max Planck Institute for Intelligent Systems T¨ubingen, Germany zjin@tue.mpg.de Geeticka Chauhan MIT geeticka@mit.edu Brian Tse Oxford briantsemanhei@gmail.com Mrinmaya Sachan ETH Z¨urich msachan@ethz.ch Rada Mihalcea University of Michigan mihalcea@umich.edu Abstract Recent years have seen many breakthroughs in natural language processing (NLP), transi- tioning it from a mostly theoretical field to one with many real-world applications. Noting the rising number of applications of other machine learning and AI techniques with pervasive so- cietal impact, we anticipate the rising impor- tance of developing NLP technologies for so- cial good. Inspired by theories in moral phi- losophy and global priorities research, we aim to promote a guideline for social good in the context of NLP. We lay the foundations via the moral philosophy definition of social good, propose a framework to evaluate the direct and indirect real-world impact of NLP tasks, and adopt the methodology of global priorities re- search to identify priority causes for NLP re- search. Finally, we use our theoretical frame- work to provide some practical guidelines for future NLP research for social good.1 1 Introduction Advances on multiple NLP fronts have given rise to a plethora of applications that are now integrated into our daily lives. NLP-based intelligent agents like Amazon Echo and Google Home have entered millions of households (Voicebot, 2020). NLP tools are now prevalent on phones, in cars, and in many daily services such as Google search and electronic health record analysis (Townsend, 2013). In the current COVID-19 context, NLP has al- ready had important positive social impact in the face of a public health crisis. When the pandemic broke out, Allen AI collected the CORD-19 dataset (Wang et al., 2020) with the goal of helping public health experts efficiently sift through the myriad of COVID-19 research papers that emerged in a short time period. Subsequently, NLP services such as 1Our data and code are available at http://github. com/zhijing-jin/nlp4sg_acl2021. Amazon Kendra were deployed to help organize the research knowledge around COVID-19 (Bhatia et al., 2020). The NLP research community worked on several problems like the question-answering and summarization system CAiRE-COVID (Su et al., 2020), the expressive interviewing conversa- tional system (Welch et al., 2020) and annotation schemas to help fight COVID-19 misinformation online (Alam et al., 2020; Hossain et al., 2020). As NLP transits from theory into practice and into daily lives, unintended negative consequences that early theoretical researchers did not anticipate have also emerged, from the toxic language of Mi- crosoft’s Twitter bot Tay (Shah and Chokkattu, 2016), to the leak of privacy of Amazon Alexa (Chung et al., 2017). A current highly-debated topic in NLP ethics is GPT-3 (Brown et al., 2020), whose risks and harms include encoding gender and racist biases (Bender et al., 2021). It is now evident that we must consider the neg- ative …

Content truncated to 500 words