GLADIS: A General and Large Acronym Disambiguation Benchmark

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2073–2088 May 2-6, 2023 ©2023 Association for Computational Linguistics GLADIS: A General and Large Acronym Disambiguation Benchmark Lihu Chen1, Gaël Varoquaux2, Fabian M. Suchanek1 1 LTCI & Télécom Paris & Institut Polytechnique de Paris, France 2 Soda, Inria Saclay, Université Paris-Saclay, France {lihu.chen, fabian.suchanek}@telecom-paris.fr {gael.varoquaux}@inria.fr Abstract Acronym Disambiguation (AD) is crucial for natural language understanding on various sources, including biomedical reports, scien- tific papers, and search engine queries. How- ever, existing acronym disambiguation bench- marks and tools are limited to specific do- mains, and the size of prior benchmarks is rather small. To accelerate the research on acronym disambiguation, we construct a new benchmark named GLADIS with three compo- nents: (1) a much larger acronym dictionary with 1.5M acronyms and 6.4M long forms; (2) a pre-training corpus with 160 million sen- tences; (3) three datasets that cover the general, scientific, and biomedical domains. We then pre-train a language model, AcroBERT, on our constructed corpus for general acronym disam- biguation, and show the challenges and values of our new benchmark. 1 Introduction An acronym is an abbreviation formed from the initial letters of a longer name. For instance, the following two sentences contain the acronym “AI”: (1) This is the product’s first true AI version, and it understands your voice instantly. (2) In the United States, the AI for potassium for adults is 4.7 grams. The long forms (or expanded forms) for the same acronym are “Artificial Intelligence” and “Ade- quate Intake”, respectively. Acronym Disambiguation (AD) is the task of mapping a given acronym in a given sentence to the intended long form. Acronym disambiguation is crucial for downstream tasks such as information extraction, machine translation, and query analysis in search engines (Jain et al., 2007; Islamaj Do- gan et al., 2009). Acronym disambiguation is also important for humans: acronyms may make a text more difficult to understand for readers who are not familiar with the specific domain. A study on a Microsoft question answering forum found that ID Long Form Popularity Domain 1 Artificial Intelligence ⋆⋆⋆⋆⋆ Computer Science 2 Adequate Intake ⋆⋆⋆⋆ Food and Nutrition 3 Aromatase Inhibitor ⋆⋆⋆ Chemistry 4 Apoptotic Index ⋆⋆⋆ Biomedicine 5 Asynchronous Irregular ⋆⋆⋆ Neuroscience 6 Amnesty International ⋆⋆ Organization 7 Anterior Insula ⋆⋆ Biomedicine 8 Air India ⋆⋆ Organization 9 Article Influence ⋆⋆ Science ...... 2243 Agricultural Implement ⋆ Agriculture Table 1: Long form candidates for the acronym “AI” from our acronym dictionary. The SciAD benchmark (Veyseh et al., 2020) only includes two long terms (black) in the scientific domain. The popularity is the occurrence frequency in our collected corpora. only 7% of the acronyms co-occur with their corre- sponding long forms, which confuses the readers about the meaning of a text (Li et al., 2018). Acronym Disambiguation has received more at- tention in the past few years. The first step in acronym disambiguation is usually the creation of a dictionary, i.e., a mapping of each acronym to …

Content truncated to 500 words