Divide and Extract – Disentangling Clause Splitting and Proposition Extraction

Proceedings of Recent Advances in Natural Language Processing, pages 399–408, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_047 399 Divide and Extract – Disentangling Clause Splitting and Proposition Extraction Darina Gold Torsten Zesch Language Technology Lab University of Duisburg-Essen, Germany {darina.gold,torsten.zesch} @uni-due.de Abstract Proposition extraction from sentences is an important task for information extraction sys- tems. Evaluation of such systems usually con- flates two aspects: splitting complex sentences into clauses and the extraction of propositions. It is thus difficult to independently determine the quality of the proposition extraction step. We create a manually annotated proposition dataset from sentences taken from restaurant reviews that distinguishes between clauses that need to be split and those that do not. The resulting proposition evaluation dataset allows us to independently compare the performance of proposition extraction systems on simple and complex clauses. Although performance drastically drops on more complex sentences, we show that the same systems perform best on both simple and complex clauses. Furthermore, we show that specific kinds of subordinate clauses pose dif- ficulties to most systems. 1 Introduction Propositions are predicate-centered tuples consist- ing of the verb, the subject, and other arguments such as objects and modifiers. For example in Figure 1, “smiled” is the predicate and the other elements are arguments. The first argument is Figure 1: Example Sentence and Extracted Proposition reserved for the role of the subject, in this case “The waitress”, while “at her friend” and “now” are arguments, without further sub-specification. Propositions are used in language understanding tasks such as relation extraction (Riedel et al., 2013; Petroni et al., 2015), information retrieval (L¨oser et al., 2011; Giri et al., 2017), question an- swering (Khot et al., 2017), word analogy detec- tion (Stanovsky et al., 2015), knowledge base con- struction (Dong et al., 2014; Stanovsky and Da- gan, 2016), summarization (Melli et al., 2006), or other tasks that need comparative operations, such as equality, entailment, or contradiction, on phrases or sentences. The main goal of this paper is to empiri- cally measure the influence of sentence complex- ity on the performance of proposition extraction systems. Complexity worsens the extraction of dependencies, on which propositions are built. Hence, proposition extraction performance should decrease with increasing sentence complexity. The contribution of this work is threefold a) a gold standard corpus for propositions1, b) an anal- ysis of proposition extraction systems without the influence of complex sentences, and c) an analysis of proposition extraction systems with the influ- ence of complex sentences. The knowledge of how proposition extraction systems perform on complex sentences will 1) help to identify the system that deals with them best 2) by showing the difficulty with complexity, give a direction towards which proposition extrac- tion systems can be improved. If different systems perform well on simple or complex sentences, the complexity distinction could help to identify the complexity of a sen- tence. The complexity of a sentence would then give a direction towards which system would be better to use. 2 Related Work Proposition are relational tupels extracted …

Content truncated to 500 words