Ordinal Common-sense Inference

Ordinal Common-sense Inference Sheng Zhang Johns Hopkins University zsheng2@jhu.edu Rachel Rudinger Johns Hopkins University rudinger@jhu.edu Kevin Duh Johns Hopkins University kevinduh@cs.jhu.edu Benjamin Van Durme Johns Hopkins University vandurme@cs.jhu.edu Abstract Humans have the capacity to draw common- sense inferences from natural language: vari- ous things that are likely but not certain to hold based on established discourse, and are rarely stated explicitly. We propose an evaluation of automated common-sense inference based on an extension of recognizing textual entail- ment: predicting ordinal human responses on the subjective likelihood of an inference hold- ing in a given context. We describe a frame- work for extracting common-sense knowledge from corpora, which is then used to construct a dataset for this ordinal entailment task. We train a neural sequence-to-sequence model on this dataset, which we use to score and gen- erate possible inferences. Further, we anno- tate subsets of previously established datasets via our ordinal annotation protocol in order to then analyze the distinctions between these and what we have constructed. 1 Introduction We use words to talk about the world. There- fore, to understand what words mean, we must have a prior explication of how we view the world. – Hobbs (1987) Researchers in Artificial Intelligence and (Compu- tational) Linguistics have long-cited the require- ment of common-sense knowledge in language un- derstanding.1 This knowledge is viewed as a key 1Schank (1975): It has been apparent ... within ... natural language understanding ... that the eventual limit to our solu- tion ... would be our ability to characterize world knowledge. Sam bought a new clock � The clock runs Dave found an axe in his garage � A car is parked in the garage Tom was accidentally shot by his teammate in the army � The teammate dies Two friends were in a heated game of checkers � A person shoots the checkers My friends and I decided to go swimming in the ocean � The ocean is carbonated Figure 1: Examples of common-sense inference ranging from very likely, likely, plausible, technically possible, to impossible. component in filling in the gaps between the tele- graphic style of natural language statements. We are able to convey considerable information in a rela- tively sparse channel, presumably owing to a par- tially shared model at the start of any discourse.2 Common-sense inference – inferences based on common-sense knowledge – is possibilistic: things everyone more or less would expect to hold in a given context, but without the necessary strength of logical entailment.3 Because natural language cor- pora exhibits human reporting bias (Gordon and Van Durme, 2013), systems that derive knowledge ex- clusively from such corpora may be more accurately considered models of language, rather than of the 2McCarthy (1959): a program has common sense if it au- tomatically deduces for itself a sufficiently wide class of imme- diate consequences of anything it is told and what it already knows. 3Many of the bridging inferences of Clark (1975) make use of common-sense knowledge, such as the following example of …

Content truncated to 500 words