Annotation of metaphorical expressions in the Basic Corpus of Polish Metaphors

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 564–573 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 564 Do we Name the Languages we Study? The #BenderRule in LREC and ACL articles Fanny Ducel†, Kar¨en Fort†⋆, Ga¨el Lejeune†, Yves Lepage‡ †STIH, Sorbonne Universit´e, France ⋆ Universit´e de Lorraine, CNRS, Inria, LORIA, France, ‡Waseda University, Japan ducelfanny@gmail.com, karen.fort@loria.fr, gael.lejeune@sorbonne-universite.fr, yves.lepage@waseda.jp Abstract This article studies the application of the #BenderRule in Natural Language Processing (NLP) articles according to two dimensions. Firstly, in a contrastive manner, by considering two major international conferences, LREC and ACL, and secondly, in a diachronic manner, by inspecting nearly 14,000 articles over a period of time ranging from 2000 to 2020 for LREC and from 1979 to 2020 for ACL. For this purpose, we created a corpus from LREC and ACL articles from the above-mentioned periods, from which we manually annotated nearly 1,000. We then developed two classifiers to automatically annotate the rest of the corpus. We show that LREC articles tend to respect the #BenderRule (80 to 90% of them respect it), whereas only around half of ACL articles do. Interestingly, over the considered periods, the results appear to be stable for the two conferences, even though a rebound in ACL 2020 could be a sign of the influence of the blog post about the #BenderRule. Keywords: #BenderRule, language diversity, ethics 1. Introduction 1.1. Background This article is positioned in the field of NLP4NLP, an acronym known from the series of articles by Mari- ani et al. (2019a) (part II in (Mariani et al., 2019b)). NLP4NLP consists in applying Natural Language Pro- cessing (NLP) techniques and exploring the field of NLP itself, by mainly inspecting the content of pub- lished articles. The aims of NLP4NLP can be various. For example, exploring the progress in the field, e.g., identifying discoveries or hot topics (Mariani et al., 2013; Mariani et al., 2014; Buitelaar et al., 2014), ad- dressing ethical issues, e.g., reuse and plagiarism (Mar- iani et al., 2016)1, or even questioning the reliability of the methods used, e.g., in machine translation (Marie et al., 2021)2. This article addresses a question with both ethical and scientific implications: we inspect the number and which languages are studied in two NLP conferences, LREC and ACL, that have already been contrasted in a previous work (Buitelaar et al., 2014). To that end, we firstly annotate a sample of articles by hand and then generalize our results to the entire col- lection of articles by using linguistic feature extraction and classification techniques from machine learning. 1.2. Topic Addressed in this Article The problem of the number and which languages are studied in the field of NLP has been raised recently, al- though it had been identified earlier in the history of grammar or linguistics (see Section 1.3). As for NLP, 1With the felicitous conclusion, at that time, that “plagia- rism is very uncommon in our community.” 2With guidelines recommending the use of more …

Content truncated to 500 words