Frame-Based Continuous Lexical Semantics through Exponential Family Tensor Factorization and Semantic Proto-Roles

Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017), pages 97–103, Vancouver, Canada, August 3-4, 2017. c⃝2017 Association for Computational Linguistics Frame-Based Continuous Lexical Semantics through Exponential Family Tensor Factorization and Semantic Proto-Roles Francis Ferraro and Adam Poliak and Ryan Cotterell and Benjamin Van Durme Center for Language and Speech Processing Johns Hopkins University {ferraro,azpoliak,ryan.cotterell,vandurme}@cs.jhu.edu Abstract We study how different frame annota- tions complement one another when learn- ing continuous lexical semantics. We learn the representations from a tensorized skip-gram model that consistently en- codes syntactic-semantic content better, with multiple 10% gains over baselines. 1 Introduction Consider “Bill” in Fig. 1: what is his involve- ment with the words “would try,” and what does this involvement mean? Word embeddings repre- sent such meaning as points in a real-valued vec- tor space (Deerwester et al., 1990; Mikolov et al., 2013). These representations are often learned by exploiting the frequency that the word cooccurs with contexts, often within a user-defined window (Harris, 1954; Turney and Pantel, 2010). When built from large-scale sources, like Wikipedia or web crawls, embeddings capture general charac- teristics of words and allow for robust downstream applications (Kim, 2014; Das et al., 2015). Frame semantics generalize word meanings to that of analyzing structured and interconnected la- beled “concepts” and abstractions (Minsky, 1974; Fillmore, 1976, 1982). These concepts, or roles, implicitly encode expected properties of that word. In a frame semantic analysis of Fig. 1, the segment “would try” triggers the ATTEMPT frame, filling the expected roles AGENT and GOAL with “Bill” and “the same tactic,” respectively. While frame semantics provide a structured form for analyzing words with crisp, categorically-labeled concepts, the encoded properties and expectations are im- plicit. What does it mean to fill a frame’s role? Semantic proto-role (SPR) theory, motivated by Dowty (1991)’s thematic proto-role theory, offers an answer to this. SPR replaces categorical roles ATTEMPT She said Bill would try the same tactic again. AGENT GOAL Figure 1: A simple frame analysis. with judgements about multiple underlying prop- erties about what is likely true of the entity fill- ing the role. For example, SPR talks about how likely it is for Bill to be a willing participant in the ATTEMPT. The answer to this and other simple judgments characterize Bill and his involvement. Since SPR both captures the likelihood of certain properties and characterizes roles as groupings of properties, we can view SPR as representing a type of continuous frame semantics. We are interested in capturing these SPR-based properties and expectations within word embed- dings. We present a method that learns frame- enriched embeddings from millions of documents that have been semantically parsed with multiple different frame analyzers (Ferraro et al., 2014). Our method leverages Cotterell et al. (2017)’s formulation of Mikolov et al. (2013)’s popular skip-gram model as exponential family principal component analysis (EPCA) and tensor factor- ization. This paper’s primary contributions are: (i) enriching learned word embeddings with mul- tiple, automatically obtained frames from large, disparate corpora; and (ii) demonstrating …

Content truncated to 500 words