Language Models for Lexical Inference in Context

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 1267–1280 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 1267 Language Models for Lexical Inference in Context Martin Schmitt1 and Hinrich Sch¨utze2 Center for Information and Language Processing (CIS) LMU Munich, Germany 1martin@cis.lmu.de 2inquiries@cislmu.org Abstract Lexical inference in context (LIiC) is the task of recognizing textual entailment between two very similar sentences, i.e., sentences that only differ in one expression. It can therefore be seen as a variant of the natural language infer- ence task that is focused on lexical semantics. We formulate and evaluate the first approaches based on pretrained language models (LMs) for this task: (i) a few-shot NLI classifier, (ii) a re- lation induction approach based on handcrafted patterns expressing the semantics of lexical in- ference, and (iii) a variant of (ii) with patterns that were automatically extracted from a cor- pus. All our approaches outperform the previ- ous state of the art, showing the potential of pretrained LMs for LIiC. In an extensive analy- sis, we investigate factors of success and failure of our three approaches.1 1 Introduction Lexical inference (LI) denotes the task of deciding whether or not an entailment relation holds between two lexical items. It is therefore related to the de- tection of other lexical relations like hyponymy between nouns (Hearst, 1992), e.g., dog ⇒ animal, or troponymy between verbs (Fellbaum and Miller, 1990), e.g., to traipse ⇒ to walk. Lexical inference in context (LIiC) adds the problem of disambiguat- ing the pair of lexical items in a given context be- fore reasoning about the inference question. This type of LI is particularly interesting for entailments between verbs and verbal expressions because their meaning – and therefore their implications – can drastically change with different arguments. Con- sider, e.g., run ⇒ lead in a PERSON / COMPANY context (“Bezos runs Amazon”) vs. run ⇒ execute in a COMPUTER / SOFTWARE context (“My mac runs macOS”). LIiC is thus also closely related to 1Our code is publicly available: https://github. com/mnschmit/lm-lexical-inference the task of natural language inference (NLI) – also called recognizing textual entailment (Dagan et al., 2013) – and can be seen as a focused variant of it. Besides the important use case of evaluating NLI systems, this kind of predicate entailment has also been shown useful for question answering (Schoen- mackers et al., 2010), event coreference (Shwartz et al., 2017; Meged et al., 2020), and link prediction in knowledge graphs (Hosseini et al., 2019). Despite its NLI nature, previous systems for LIiC have primarily been models of lexical sim- ilarity (Levy and Dagan, 2016) or models based on verb argument inclusion (Hosseini et al., 2019). The reason is probably that supervised NLI models need large amounts of training data, which is un- available for LIiC, and that systems trained on avail- able large-scale NLI benchmarks (e.g., Williams et al., 2018) have been reported to insufficiently cover lexical phenomena (Glockner et al., 2018; Schmitt and Sch¨utze, …

Content truncated to 500 words