Coverage of Information Extraction from Sentences and Paragraphs

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5771–5776, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5771 Coverage of Information Extraction from Sentences and Paragraphs Simon Razniewski1, Nitisha Jain2, Paramita Mirza1, Gerhard Weikum1 1 Max Planck Institute for Informatics 2 Hasso Plattner Institute {srazniew, paramita, weikum}@mpi-inf.mpg.de nitisha.jain@hpi.de Abstract Scalar implicatures are language features that imply the negation of stronger statements, e.g., “She was married twice” typically implicates that she was not married thrice. In this paper we discuss the importance of scalar implica- tures in the context of textual information ex- traction. We investigate how textual features can be used to predict whether a given text seg- ment mentions all objects standing in a cer- tain relationship with a certain subject. Pre- liminary results on Wikipedia indicate that this prediction is feasible, and yields informative assessments. 1 Introduction Following the cooperative principle, natural lan- guage utterances can implicate a range of asser- tions that are not explicitly stated (Grice, 1975). One specific class of implicatures are scalar im- plicatures, which concern the negation of stronger statements (Carston, 1998). Scalar implicatures are derived from Grice’s maxim of quantity - that speakers would make stronger statements if possi- ble, therefore, negation can be deduced if these are not made. Yet the maxim of quantity interacts with the maxim of relevance, i.e., what is implicated de- pends on what is relevant in a given context. Con- sider the examples in Figure 1. From the first sen- tence, typical for biographical descriptions, most humans would draw the implicature that Obama has no other children. For the second sentence about Jolie, this implicature would typically not be drawn - she might well have other children that are too young or too old to be brought to school, but are not relevant in the school context. The interaction between the maxims of quan- tity and relevance has implications for textual in- formation extraction (IE). Textual IE usually pro- duces (ideally canonicalized) subject-predicate- Figure 1: Two sentences with different coverage. object triples, and annotates them with estimates of correctness (also called accuracy, precision or confidence), e.g., a 93% belief that Malia is really Obama’s child. In contrast, IE usually lacks such an ability for coverage (also called recall or com- pleteness). It is not able to estimate whether its extractions represent all facts pertaining to a cer- tain topic, e.g., all children of Jolie. Importance of coverage-awareness. Coverage- awareness of IE is a crucial and highly desir- able property for several downstream use cases. (i) Today’s question answering systems are well geared for questions where exactly one answer should be returned (e.g., quiz questions or reading- comprehension tasks) (Fader et al., 2014; Yang et al., 2015). In contrast, for questions with sets of answers, QA systems often merely yield subsets, and do not inform the user about that. Similarly, they struggle with questions that have no answer, too …

Content truncated to 500 words