Learning Semantically and Additively Compositional Distributional Representations

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1277–1287, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Learning Semantically and Additively Compositional Distributional Representations Ran Tian and Naoaki Okazaki and Kentaro Inui Tohoku University, Japan {tianran, okazaki, inui}@ecei.tohoku.ac.jp Abstract This paper connects a vector-based com- position model to a formal semantics, the Dependency-based Compositional Se- mantics (DCS). We show theoretical evi- dence that the vector compositions in our model conform to the logic of DCS. Ex- perimentally, we show that vector-based composition brings a strong ability to calculate similar phrases as similar vec- tors, achieving near state-of-the-art on a wide range of phrase similarity tasks and relation classification; meanwhile, DCS can guide building vectors for structured queries that can be directly executed. We evaluate this utility on sentence comple- tion task and report a new state-of-the-art. 1 Introduction A major goal of semantic processing is to map nat- ural language utterances to representations that fa- cilitate calculation of meanings, execution of com- mands, and/or inference of knowledge. Formal semantics supports such representations by defin- ing words as some functional units and combining them via a specific logic. A simple and illustra- tive example is the Dependency-based Composi- tional Semantics (DCS) (Liang et al., 2013). DCS composes meanings from denotations of words (i.e. sets of things to which the words apply); say, the denotations of the concept drug and the event ban is shown in Figure 1b, where drug is a list of drug names and ban is a list of the subject- complement pairs in any ban event; then, a list of banned drugs can be constructed by first taking the COMP column of all records in ban (projection “πCOMP”), and then intersecting the results with drug (intersection “∩”). This procedure defined how words can be combined to form a meaning. Better yet, the procedure can be concisely illus- trated by the DCS tree of “banned drugs” (Fig- ure 1a), which is similar to a dependency tree but possesses precise procedural and logical meaning (Section 2). DCS has been shown useful in ques- tion answering (Liang et al., 2013) and textual en- tailment recognition (Tian et al., 2014). Orthogonal to the formal semantics of DCS, distributional vector representations are useful in capturing lexical semantics of words (Turney and Pantel, 2010; Levy et al., 2015), and progress is made in combining the word vectors to form meanings of phrases/sentences (Mitchell and La- pata, 2010; Baroni and Zamparelli, 2010; Grefen- stette and Sadrzadeh, 2011; Socher et al., 2012; Paperno et al., 2014; Hashimoto et al., 2014). However, less effort is devoted to finding a link between vector-based compositions and the com- position operations in any formal semantics. We believe that if a link can be found, then symbolic formulas in the formal semantics will be realized by vectors composed from word embeddings, such that similar things are realized by similar vectors; meanwhile, vectors will acquire formal meanings that can directly be used in execution or …

Content truncated to 500 words