The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7484 - 7499 May 22-27, 2022 c⃝2022 Association for Computational Linguistics The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail Samuel R. Bowman New York University bowman@nyu.edu Abstract Researchers in NLP often frame and discuss research results in ways that serve to deem- phasize the field’s successes, often in response to the field’s widespread hype. Though well- meaning, this has yielded many misleading or false claims about the limits of our best tech- nology. This is a problem, and it may be more serious than it looks: It harms our credibil- ity in ways that can make it harder to miti- gate present-day harms, like those involving bi- ased systems for content moderation or resume screening. It also limits our ability to prepare for the potentially enormous impacts of more distant future advances. This paper urges re- searchers to be careful about these claims and suggests some research directions and commu- nication strategies that will make it easier to avoid or rebut them. 1 Introduction Over the last few years, natural language process- ing has seen a wave of surprising negative re- sults overturning previously-reported success sto- ries about what our models can do, and showing that widely-used models are surprisingly brittle (Jia and Liang, 2017; Niven and Kao, 2019; McCoy et al., 2019). This shows that many of our standard practices for evaluation and reporting can lead to unrealistically positive initial claims about what we can do. The resulting hype and overclaiming, whether intentional or not, are a problem. They can encourage the reckless deployment of NLP systems in high-stakes settings where they can do significant harm. They also threaten the health and credibility of NLP as a research field, and thereby threaten our ability to influence applied stakehold- ers or attract funding. Fortunately, these results have led to a surge of research and writing that proposes more thorough and cautious practices for the evaluation of model ability (Ribeiro et al., 2020; Gardner et al., 2020; Figure 1: Hype is a problem. The opposite of hype isn’t necessarily better. (Quoted with permission.) Kiela et al., 2021; Bowman and Dahl, 2021). While we have only a limited ability to control the pub- lic narrative taking place through industry PR and the media, there’s reason to be hopeful that we re- searchers are getting much better at avoiding the worst forms of overconfidence about our systems. Less fortunately, this pattern of disappointment seems to have led to many instances of pessimism about model performance that are ungrounded from real empirical results. This leaves room for the research community’s consensus about our capabil- ities to fall short of our actual capabilities. I call this issue underclaiming, for lack of a bet- ter term,1 and argue that it is more dangerous than it might seem. It risks our credibility and thereby limits our ability to influence stakeholders in cases where our current …

Content truncated to 500 words