Composing Distributed Representations of Relational Patterns

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 2276–2286, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Composing Distributed Representations of Relational Patterns Sho Takase Naoaki Okazaki Kentaro Inui Graduate School of Information Sciences, Tohoku University {takase, okazaki, inui}@ecei.tohoku.ac.jp Abstract Learning distributed representations for relation instances is a central technique in downstream NLP applications. In or- der to address semantic modeling of rela- tional patterns, this paper constructs a new dataset that provides multiple similarity ratings for every pair of relational patterns on the existing dataset (Zeichner et al., 2012). In addition, we conduct a compar- ative study of different encoders includ- ing additive composition, RNN, LSTM, and GRU for composing distributed rep- resentations of relational patterns. We also present Gated Additive Composition, which is an enhancement of additive com- position with the gating mechanism. Ex- periments show that the new dataset does not only enable detailed analyses of the different encoders, but also provides a gauge to predict successes of distributed representations of relational patterns in the relation classification task. 1 Introduction Knowledge about entities and their relations (re- lation instances) are crucial for a wide spectrum of NLP applications, e.g., information retrieval, question answering, and recognizing textual en- tailment. Learning distributed representations for relation instances is a central technique in down- stream applications as a number of recent studies demonstrated the usefulness of distributed repre- sentations for words (Mikolov et al., 2013; Pen- nington et al., 2014) and sentences (Sutskever et al., 2014; Cho et al., 2014; Kiros et al., 2015). In particular, semantic modeling of relations and their textual realizations (relational patterns hereafter) is extremely important because a rela- Zeichner+ (2012) (increase the risk of, cause): (be open from, close at): ......... 5 6 6 6 7 1 1 2 2 2 .......... Similarity ratings of pattern pairs increase: risk: open: increase the risk of: cause: Corpus (ukWaC) ... evidence that passive smoking increases the risk of lung cancer by 10%. The research also ... ...... Cigarette smoking causes breathing problems ... Usually, pubs closes at midnight, and people increase the risk of passive smoking lung cancer cause cigarette smoking heart attack close at pub midnight be open from department store 10am Relational pattern X Y ... ... ... Encoder for relational patterns (§3) Additive composition, RNN, LSTM, GRU, GAC be open from: Annotating human judgments using crowd sourcing (§2) Embeddings of relational patterns Word embeddings SemEval 2010 Task 8 (word2vec) Training Open IE (Reverb) Relational pattern similarity Evaluation (§4.1) Relation classification Evaluation (§4.2) Figure 1: Overview of this study. tion (e.g., causality) can be mentioned by various expressions (e.g., “X cause Y”, “X lead to Y”, “Y is associated with X”). To make matters worse, rela- tional patterns are highly productive: we can pro- duce a emphasized causality pattern “X increase the severe risk of Y” from “X increase the risk of Y” by inserting severe to the pattern. To model the meanings of relational patterns, the previous studies …

Content truncated to 500 words