Functional Distributional Semantics at Scale

Proceedings of the The 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023), pages 423–436 July 13-14, 2023 ©2023 Association for Computational Linguistics Functional Distributional Semantics at Scale Chun Hei Lo1 Hong Cheng1 Wai Lam1 Guy Emerson2 1The Chinese University of Hong Kong {chlo, hcheng, wlam}@se.cuhk.edu.hk2University of Cambridge gete2@cam.ac.uk Abstract Functional Distributional Semantics is a lin- guistically motivated framework for modelling lexical and sentence-level semantics with truth- conditional functions using distributional infor- mation. Previous implementations of the frame- work focus on subject–verb–object (SVO) triples only, which largely limits the contex- tual information available for training and thus the capability of the learnt model. In this pa- per, we discuss the challenges of extending the previous architectures to training on arbi- trary sentences. We address the challenges by proposing a more expressive lexical model that works over a continuous semantic space. This improves the flexibility and computational effi- ciency of the model, as well as its compatibility with present-day machine-learning frameworks. Our proposal allows the model to be applied to a wider range of semantic tasks, and improved performances are demonstrated from experi- mental results. 1 Introduction Functional Distributional Semantics (FDS; Emer- son and Copestake, 2016; Emerson, 2018) aims to capture the truth-conditional aspects of words through learning from distributional information of a corpus. Whilst truth-conditional semantics deals with predications over discrete entities, FDS aims to generalize about predications over a space of entity representations with probabilistic semantics. Contrasted with most distributional methods which map words to vectors, FDS can model vari- ous aspects of meaning in a linguistically rigorous manner. For example, vagueness is represented by the probabilistic nature of predications, and hy- pernymy, defined formally as the subsumption of the extensions between two word senses, can be represented by the subsumption of regions of space (Emerson, 2020b). Going beyond simple vector spaces, some mod- els of distributional semantics represent words as tensors for composition (e.g., Coecke et al., 2010; Baroni et al., 2014), as static distributions for un- certainty and entailment (e.g., Vilnis and McCal- lum, 2015), as posterior distributions for context- specific meaning (e.g., Bražinskas et al., 2018), and as regions for set-theoretic properties (e.g., Das- gupta et al., 2022). Among them, only a region- based approach favours logical interpretations (for a discussion, see: Emerson, 2020b, 2023). In order to be computationally tractable, most models of distributional semantics are trained based on instances defined by context windows (e.g., Mikolov et al., 2013a; Pennington et al., 2014) or incomplete linguistic structures such as immedi- ate dependencies (e.g., Levy and Goldberg, 2014; Czarnowska et al., 2019). All previous instances of FDS (further discussed in §2) are only trained on SVO triples. Consequently, these models underuti- lize much contextual information. We hope to extend FDS learning to arbitrary sen- tences, but not larger linguistic units (e.g., para- graphs), for handling them requires non-trivial extensions such as robust coreference resolution, which is beyond the scope of this work. To this end, we propose to adopt a continuous semantic space and a …

Content truncated to 500 words