Classifying Semantic Clause Types: Modeling Context and Genre Characteristics with Recurrent Neural Networks and Attention

Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017), pages 230–240, Vancouver, Canada, August 3-4, 2017. c⃝2017 Association for Computational Linguistics Classifying Semantic Clause Types: Modeling Context and Genre Characteristics with Recurrent Neural Networks and Attention Maria Becker♦♠, Michael Staniek♦♠, Vivi Nastase♦♠, Alexis Palmer♣, Anette Frank♦♠ ♦ Leibniz ScienceCampus “Empirical Linguistics and Computational Language Modeling” ♠Heidelberg University, Department of Computational Linguistics ♣University of North Texas, Department of Linguistics {mbecker,staniek,nastase,frank}@cl.uni-heidelberg.de alexis.palmer@unt.edu Abstract Detecting aspectual properties of clauses in the form of situation entity types has been shown to depend on a combination of syntactic-semantic and contextual fea- tures. We explore this task in a deep- learning framework, where tuned word representations capture lexical, syntactic and semantic features. We introduce an attention mechanism that pinpoints rele- vant context not only for the current in- stance, but also for the larger context. Apart from implicitly capturing task rel- evant features, the advantage of our neu- ral model is that it avoids the need to re- produce linguistic features for other lan- guages and is thus more easily transfer- able. We present experiments for English and German that achieve competitive per- formance. We present a novel take on modeling and exploiting genre informa- tion and showcase the adaptation of our system from one language to another. 1 Introduction Semantic clause types, called Situation Entity (SE) types (Smith, 2003; Palmer et al., 2007) are lin- guistic characterizations of aspectual properties shown to be useful for argumentation structure analysis (Becker et al., 2016b), genre characteriza- tion (Palmer and Friedrich, 2014), and detection of generic and generalizing sentences (Friedrich and Pinkal, 2015). Recent work on automatic identi- fication of SE types relies on feature-based classi- fiers for English that have been successfully ap- plied to various textual genres (Friedrich et al., 2016), and also show that a sequence labeling ap- proach that models contextual clause labels yields improved classification performance. Deep learning provides a powerful framework in which linguistic and semantic regularities can be implicitly captured through word embeddings (Mikolov et al., 2013b). Patterns in larger text fragments can be encoded and exploited by re- current (RNNs) or convolutional neural networks (CNNs) which have been successfully used for various sentence-based classification tasks, e.g. sentiment (Kim, 2014) or relation classification (Vu et al., 2016; Tai et al., 2015). We frame the task of classifying clauses with re- spect to their aspectual properties – i.e., situation entity types – in a recurrent neural network archi- tecture. We adopt a Gated Recurrent Unit (GRU)- based RNN architecture that is well suited to mod- eling long sequences (Yin et al., 2017). This ini- tial model is enhanced with an attention mecha- nism shown to be beneficial for sentence classifi- cation (Wang et al., 2016) and sequence modeling (Dong and Lapata, 2016). We explore the useful- ness of attention in two settings: (i) the individ- ual classification task and (ii) in a setting approx- imating sequential labeling in which the attention vector provides features that describe the clauses …

Content truncated to 500 words