Mining Knowledge for Natural Language Inference from Wikipedia Categories

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3500–3511 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 3500 Mining Knowledge for Natural Language Inference from Wikipedia Categories Mingda Chen3∗ Zewei Chu2∗ Karl Stratos1 Kevin Gimpel3 1Rutgers University, NJ, USA 2University of Chicago, IL, USA 3Toyota Technological Institute at Chicago, IL, USA {mchen,kgimpel}@ttic.edu,zeweichu@gmail.com,stratos@cs.rutgers.edu Abstract Accurate lexical entailment (LE) and natural language inference (NLI) often require large quantities of costly annotations. To allevi- ate the need for labeled data, we introduce WIKINLI: a resource for improving model performance on NLI and LE tasks. It con- tains 428,899 pairs of phrases constructed from naturally annotated category hierarchies in Wikipedia. We show that we can improve strong baselines such as BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019) by pre- training them on WIKINLI and transferring the models on downstream tasks. We con- duct systematic comparisons with phrases ex- tracted from other knowledge bases such as WordNet and Wikidata to find that pretrain- ing on WIKINLI gives the best performance. In addition, we construct WIKINLI in other languages, and show that pretraining on them improves performance on NLI tasks of corre- sponding languages.1 1 Introduction Natural language inference (NLI) is the task of classifying the relationship, such as entailment or contradiction, between sentences. It has been found useful in downstream tasks, such as summarization (Mehdad et al., 2013) and long-form text gener- ation (Holtzman et al., 2018). NLI involves rich natural language understanding capabilities, many of which relate to world knowledge. To acquire such knowledge, researchers have found benefit from external knowledge bases like WordNet (Fell- baum, 1998), FrameNet (Baker, 2014), Wikidata (Vrandeˇci´c and Kr¨otzsch, 2014), and large-scale human-annotated datasets (Bowman et al., 2015; Williams et al., 2018; Nie et al., 2020). Creating ∗Equal contribution. Listed in alphabetical order. 1Code and data are available at https://github. com/ZeweiChu/WikiNLI. these resources generally requires expensive hu- man annotation. In this work, we are interested in automatically generating a large-scale dataset from Wikipedia categories that can improve performance on both NLI and lexical entailment (LE) tasks. One key component of NLI tasks is recognizing lexical and phrasal hypernym relationships. For example, vehicle is a hypernym of car. In this pa- per, we take advantage of the naturally-annotated Wikipedia category graph, where we observe that most of the parent-child category pairs are entail- ment relationships, i.e., a child category entails a parent category. Compared to WordNet and Wikidata, the Wikipedia category graph has more fine-grained connections, which could be help- ful for training models. Inspired by this observa- tion, we construct WIKINLI, a dataset for training NLI models constructed automatically from the Wikipedia category graph, by automatic filtering from the Wikipedia category graph. The dataset has 428,899 pairs of phrases and contains three categories that correspond to the entailment and neutral relationships in NLI datasets. To empirically demonstrate the usefulness of WIKINLI, we pretrain BERT and RoBERTa on WIKINLI, WordNet, and Wikidata, before finetun- ing on various LE and NLI tasks. Our …

Content truncated to 500 words