Clarifying Implicit and Underspecified Phrases in Instructional Text

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 3319–3330 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 3319 Clarifying Implicit and Underspecified Phrases in Instructional Text Talita Anthonio∗ Anna Sauer∗ Michael Roth University of Stuttgart Institute for Natural Language Processing {talita.anthonio,anna.sauer,michael.roth}@ims.uni-stuttgart.de Abstract Natural language inherently consists of implicit and underspecified phrases, which represent potential sources of misunder- standing. In this paper, we present a data set of such phrases in English from instructional texts together with multiple possible clarifications. Our data set, henceforth called CLAIRE, is based on a corpus of revision histories from wikiHow, from which we extract human clarifications that resolve an implicit or underspecified phrase. We show how language modeling can be used to generate alternate clarifications, which may or may not be compatible with the human clarification. Based on plausibility judgements for each clarification, we define the task of distinguishing between plausible and implausible clarifications. We provide several baseline models for this task and analyze to what extent different clarifications represent multiple interpretations as a first step to investigate misunderstandings caused by implicit/underspecified language in instructional texts. Keywords: Corpus, Language Modeling, Semantics, Other 1. Introduction Natural language inherently consists of elements that are implicit or underspecified because it is assumed that they can be inferred by the reader. For example, anaphoric references are often omitted when the refer- ent is clear from the discourse context (Fillmore, 1986, henceforth implicit references). Similarly, relevant as- pects of meaning may also be unspecified in explicit references, as is the case of pronominal or metonymic referring expressions. As a consequence, inferences can be difficult for a reader when different interpre- tations seem plausible. For instance, consider the in- structional text in Table 1. Here, different clarifications could be made to specify the implied information, rep- resented by the blank , as terms such as body pos- ture, walking posture and core posture could all make sense in the given context. However, these clarifica- tions are not semantically equivalent. Various NLP tasks deal with settings in which mod- els have to predict the most likely continuation of a text, such as the story cloze task (Mostafazadeh et al., 2016), the LAMBADA word prediction test (Paperno et al., 2016), or the HellaSwag sentence completion task (Zellers et al., 2018). Models in such tasks are typ- ically evaluated by their capability of detecting the cor- rect filler. This ignores the notion that different com- pletions could be plausible, as discussed by Anthonio and Roth (2021) for the case of implicit references. To bridge this gap, we believe that it is important to cre- ate resources that accurately reflect that multiple plau- sible completions or clarifications may exist for im- plicit/underspecified elements in language. Another contribution of such a resource would be to fa- cilitate research on misunderstandings that arise from texts, which has received little attention in our commu- nity so far. Misunderstandings can be viewed as diverg- ∗ The first two authors …

Content truncated to 500 words