Problem-solving Recognition in Scientific Text

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6045–6058 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6045 Problem-solving Recognition in Scientific Text Kevin Heffernan, Simone Teufel University of Cambridge Dept. of Computer Science and Technology 15 JJ Thomson Ave, Cambridge CB3 0FD {firstname.lastname}@cst.cam.ac.uk Abstract As far back as Aristotle, problems and solutions have been recognised as a core pattern of thought, and in particular of the scientific method. In this work, we present the novel task of problem-solving recognition in scientific text. Previous work on problem-solving either is not computational, is not adapted to scientific text, or has been narrow in scope. This work provides a new annotation scheme of problem-solving tailored to the scientific domain. We validate the scheme with an annotation study, and model the task using state-of-the-art baselines such as a Neural Relational Topic Model. The agreement study indicates that our annotation is reliable, and results from modelling show that problem-solving expressions in text can be recognised to a high degree of accuracy. Keywords: Problem-solving, Information Extraction, Machine Learning, Neural Relational Topic Model 1. Introduction In the field of cognitive psychology, problem-solving is formally defined as: “cognitive processing directed at achieving a goal for which the problem solver does not initially know a solution method” (Reisberg and Mayer, 2013). Many of us perform it on a daily basis, whether it be deciding on the best route home, what meal to cook for dinner, or even how to structure our day. It is generally regarded as the most important cognitive activity in everyday and professional contexts (Jonassen, 2000). According to Jordan (1980), this activity carries over from everyday activities, to any text produced. In par- ticular, there is a close connection to scientific writ- ing because the nature of the research process can be viewed as a problem-solving activity (Popper, 1999; Str¨ubing, 2007). Therefore, problem-solving plays a significant role in the understanding of academic texts from the scientific domain, and many descriptions re- lating to problems and solutions can be found in scien- tific texts. Consider the following extract, taken from Benotti and Denis (2011). Semantic annotation and rule authoring have long been known as bottlenecks for developing conversational systems for new domains. In this paper, we present a novel algorithm for gener- ating virtual instructors from automatically annotated human-human corpora. In the above extract, the first sentence details a prob- lem. The authors then introduce an algorithm which is able to use automatically annotated texts and thus solve the problem of expensive manual anntotation. In this work, we set out to automatically identify such problems and their corresponding solutions in scientific documents. Capturing knowledge of such problem-solving expressions would provide a deep in- sight into text understanding, but problem-solving is a non-trivial subjective task. For these kinds of tasks, particularly if they are newly defined, there is general consensus that careful human annotation is necessary. We developed a scheme of problem-solving, tai- lored to the scientific …

Content truncated to 500 words