Building Dataset for Grounding of Formulae — Annotating Coreference Relations Among Math Identifiers

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4851–4858 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4851 Building Dataset for Grounding of Formulae — Annotating Coreference Relations Among Math Identifiers Takuto Asakura1, Yusuke Miyao1, Akiko Aizawa1,2 1Dept. of Computer Science, The University of Tokyo, 2National Institute of Informatics, Tokyo, Japan {takuto,yusuke}@is.s.u-tokyo.ac.jp, aizawa@nii.ac.jp Abstract Grounding the meaning of each symbol in math formulae is important for automated understanding of scientific documents. Generally speaking, the meanings of math symbols are not necessarily constant, and the same symbol is used in multiple meanings. Therefore, coreference relations between symbols need to be identified for grounding, and the task has aspects of both description alignment and coreference analysis. In this study, we annotated 15 papers selected from arXiv.org with the grounding information. In total, 12,352 occurrences of math identifiers in these papers were annotated, and all coreference relations between them were made explicit in each paper. The constructed dataset shows that regardless of the ambiguity of symbols in math formulae, coreference relations can be labeled with a high inter-annotator agreement. The constructed dataset enables us to achieve automation of formula grounding, and in turn, make deeper use of the knowledge in scientific documents using techniques such as math information extraction. The built grounding dataset is available at https://sigmathling. kwarc.info/resources/grounding-dataset/. Keywords: Math Linguistics, Math Information Retrieval (MathIR), Coreference Relations, Annotation Tool 1. Introduction Understanding math formulae is as important as under- standing natural language texts to analyze documents in science, technology, engineering, and mathematics. Analyzing math formulae is unavoidable to fully exploit the knowledge contained in scientific documents by us- ing applied technology in computer science such as in- formation retrieval, computer algebra systems, and the- orem proving. In order to understand a math formula in documents, it is necessary to clarify the meaning of each formula token, that is, a character or symbol that appears in the math formula. This part of formula analysis is formalized as a task of formula grounding (Asakura et al., 2020). The grounding task has two characteristics: one is the description alignment task, which assigns a context-specific description to each formula token (Fig- ure 1), and the other is the coreference resolution task, which discriminates tokens that are used with exactly the same meaning from those that are not. In order to automate the process of formula ground- ing, the authors first worked on constructing a corpus with ground truth annotations manually for observa- tion, analysis, learning, and evaluation. As annotation of coreference information is generally costly (Oberle, 2018), the authors developed a special annotation tool, MioGatto1, to streamline the data construction pro- cess (Asakura et al., 2021). The authors then used MioGatto to annotate a total of 15 scientific papers with 11 student annotators for all occurrences of math iden- tifiers in the papers. In this paper, we introduce the procedure of constructing a dataset of formula grounding and report an overview of the constructed annotated corpus. 1https://github.com/wtsnjp/MioGatto …

Content truncated to 500 words