NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning

Proceedings of the 10th Conference on Lexical and Computational Semantics, pages 78–88 August 5–6, 2021, Bangkok, Thailand (online) ©2021 Association for Computational Linguistics 78 NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning Zeming Chen†∗ Qiyue Gao† Lawrence S. Moss‡ †Rose-Hulman Institute of Technology, Terre Haute, IN, USA ‡Indiana University, Bloomington, IN, USA {chenz16,gaoq}@rose-hulman.edu {lmoss}@indiana.edu Abstract Deep learning (DL) based language models achieve high performance on various bench- marks for Natural Language Inference (NLI). And at this time, symbolic approaches to NLI are receiving less attention. Both approaches (symbolic and DL) have their advantages and weaknesses. However, currently, no method combines them in a system to solve the task of NLI. To merge symbolic and deep learn- ing methods, we propose an inference frame- work called NeuralLog, which utilizes both a monotonicity-based logical inference engine and a neural network language model for phrase alignment. Our framework models the NLI task as a classic search problem and uses the beam search algorithm to search for opti- mal inference paths. Experiments show that our joint logic and neural inference system improves accuracy on the NLI task and can achieve state-of-art accuracy on the SICK and MED datasets. 1 Introduction Currently, many NLI benchmarks’ state-of-the-art systems are exclusively deep learning (DL) based language models (Devlin et al., 2019; Lan et al., 2020; Liu et al., 2020; Yin and Schütze, 2017). These models often contain a large number of pa- rameters, use high-quality pre-trained embeddings, and are trained on large-scale datasets, which en- able them to handle diverse and large test data ro- bustly. However, several experiments show that DL models lack generalization ability, adopt fallible syntactic heuristics, and show exploitation of anno- tation artifacts (Glockner et al., 2018; McCoy et al., 2019; Gururangan et al., 2018). On the other hand, there are logic-based systems that use symbolic reasoning and semantic formalism to solve NLI (Abzianidze, 2017; Martínez-Gómez et al., 2017; *The first two authors have equal contribution Figure 1: Analogy between path planning and an entail- ment inference path from the premise A motorcyclist with a red helmet is riding a blue motorcycle down the road to the hypothesis A motorcyclist is riding a motor- bike along a roadway. Yanaka et al., 2018; Hu et al., 2020). These systems show high precision on complex inferences involv- ing difficult linguistic phenomena and present logi- cal and explainable reasoning processes. However, these systems lack background knowledge and do not handle sentences with syntactic variations well, which makes them poor competitors with state-of- the-art DL models. Both DL and logic-based sys- tems show a major issue with NLI models: they are too one-dimensional (either purely DL or purely logic), and no method has combined these two ap- 79 proaches together for solving NLI. This paper makes several contributions, as fol- lows: first, we propose a new framework in section 3 for combining logic-based inference with deep- learning-based network inference for better perfor- mance on conducting natural language inference. We model an NLI task as a …

Content truncated to 500 words