STAR: Cross-modal [STA]tement [R]epresentation for selecting relevant mathematical premises

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 3234–3243 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 3234 STAR: Cross-modal STAtement Representation for selecting relevant mathematical premises Deborah Ferreira1, Andr´e Freitas1,2 1The University of Manchester, Department of Computer Science, Manchester, UK 2Idiap Research Institute, Switzerland {deborah.ferreira, andre.freitas}@manchester.ac.uk Abstract Mathematical statements written in natural lan- guage are usually composed of two different modalities: mathematical elements and natu- ral language. These two modalities have sev- eral distinct linguistic and semantic properties. State-of-the-art representation techniques have demonstrated an inability in capturing such an entangled style of discourse. In this work, we propose STAR, a model that uses cross- modal attention to learn how to represent math- ematical text for the task of Natural Language Premise Selection. This task uses conjectures written in both natural and mathematical lan- guage to recommend premises that most likely will be relevant to prove a particular statement. We found that STAR not only outperforms baselines that do not distinguish between natu- ral language and mathematical elements, but it also achieves better performance than state-of- the-art models. 1 Introduction Natural language understanding has been applied to several different tasks and areas, from question answering to visual grounding. Even though Math- ematics is a well-established field with immense importance for most areas of science, applications of NLP in this field are still limited. Natural language premise selection (NLPS) (Fer- reira and Freitas, 2020a) is a task that requires the combination of natural language reasoning and mathematical reasoning. Given a certain conjec- ture (a mathematical statement written in natural language) that needs to be proven, we attempt to recommend useful premises that can be relevant for developing that mathematical argument. Mathematical statements have a particular dis- course structure that makes it challenging to use traditional NLP techniques. Some of its distinc- tive features are: (1) Entangled dual lexical spaces for the mathematical elements (ME) and natural language (NL); (2) Distinct syntactic phenomena between ME and NL. Given this entangled nature of the discourse, where two very different linguistic modalities co- exist in the same text, traditional information re- trieval approaches are not able to capture the dif- ferent semantics for each modality (Greiner-Petter et al., 2019). For example, in the mathematical domain, variables are represented using generic symbols; this lexical layer does not necessarily ground the semantics of the variables. The context surrounding the variables is more important than the symbol itself. When interpreting mathematical discourse, such particulars need to be taken into account. In this work, we propose STAR, a cross-modal representation for mathematical statements for ad- dressing the task of premise selection. In order to interpret the different modalities in the mathemat- ical discourse (natural language and equational), STAR uses two different self-attention layers, one focused on the mathematical elements, such as ex- pressions and variables, while the other attends to natural language features. STAR is taught to see these tokens as parts of different languages, the mathematical language …

Content truncated to 500 words