What are the Goals of Distributional Semantics?

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7436–7453 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 7436 What are the Goals of Distributional Semantics? Guy Emerson Department of Computer Science and Technology University of Cambridge gete2@cam.ac.uk Abstract Distributional semantic models have become a mainstay in NLP, providing useful features for downstream tasks. However, assessing long-term progress requires explicit long-term goals. In this paper, I take a broad linguistic perspective, looking at how well current mod- els can deal with various semantic challenges. Given stark differences between models pro- posed in different subfields, a broad perspec- tive is needed to see how we could integrate them. I conclude that, while linguistic insights can guide the design of model architectures, fu- ture progress will require balancing the often conflicting demands of linguistic expressive- ness and computational tractability. 1 Introduction In order to assess progress in any field, the goals need to be clear. In assessing progress in semantics, Koller (2016) contrasts “top-down” and “bottom- up” approaches: a top-down approach begins with an overarching goal, and tries to build a model to reach it; a bottom-up approach begins with exist- ing models, and tries to extend them towards new goals.1 Like much of NLP, distributional seman- tics is largely bottom-up: the goals are usually to improve performance on particular tasks, or par- ticular datasets. Aiming to improve NLP applica- tions is of course a legitimate decision, but Koller points out a problem if there is no top-down goal: “Bottom-up theories are intrinsically unfalsifiable... We won’t know where distributional semantics is going until it has a top-down element”. This is contrasted against truth-conditional semantics, a traditional linguistic approach which is largely top- down: “truth-conditional semantics hasn’t reached its goal, but at least we knew what the goal was”. In this paper, I take a long-term linguistic per- spective, where the top-down goal is to characterise 1For further discussion, see: Bender and Koller (2020). the meanings of all utterances in a language. This is an ambitious goal, and a broad one. To make this goal more precise, in the following sections I will elaborate on several aspects of meaning which could be considered crucial. For each aspect, I identify a plausible goal, lay out out the space of possible models, place existing work in this space, and evaluate which approaches seem most promis- ing. By making the goals explicit, we can assess whether we are heading in the right direction, and we can assess what still needs to be done. If a reader should disagree with my conclusions, they should start by looking at my goals. 2 Background: Distributional Semantics The aim of distributional semantics is to learn the meanings of linguistic expressions from a corpus of text. The core idea, known as the distributional hy- pothesis, is that the contexts in which an expression appears give us information about its meaning.2 The idea has roots in American structuralism (Harris, 1954) and British lexicology (Firth, 1951, 1957)3, …

Content truncated to 500 words