Modeling Sense Structure in Word Usage Graphs with the Weighted Stochastic Block Model

Proceedings of the 10th Conference on Lexical and Computational Semantics, pages 241–251 August 5–6, 2021, Bangkok, Thailand (online) ©2021 Association for Computational Linguistics 241 Modeling Sense Structure in Word Usage Graphs with the Weighted Stochastic Block Model Dominik Schlechtweg, Enrique Castaneda, Jonas Kuhn, Sabine Schulte im Walde Institute for Natural Language Processing, University of Stuttgart {schlecdk,jonas.kuhn,schulte}@ims.uni-stuttgart.de, kicasta@gmail.com Abstract We suggest to model human-annotated Word Usage Graphs capturing fine-grained seman- tic proximity distinctions between word uses with a Bayesian formulation of the Weighted Stochastic Block Model, a generative model for random graphs popular in biology, physics and social sciences. By providing a probabilis- tic model of graded word meaning we aim to approach the slippery and yet widely used no- tion of word sense in a novel way. The pro- posed framework enables us to rigorously com- pare models of word senses with respect to their fit to the data. We perform extensive ex- periments and select the empirically most ade- quate model. 1 Introduction Word Usage Graphs (WUGs) are a relatively new model of graded word meaning in context (Erk et al., 2013; McCarthy et al., 2016; Schlechtweg et al., 2021). They represent word uses (i.e., words in context) within a weighted undirected graph, with edge weights reflecting the semantic proxim- ity between uses. WUGs may be obtained via hu- man annotation by presenting annotators with pairs of words uses and asking them for proximity judg- ments. The WUGs may then be clustered into sets of uses exhibiting high semantic proximity, in order to reflect traditional word sense distinctions (Mc- Carthy et al., 2016), and to provide insight into key aspects of word meaning such as polysemy, vague- ness, and lexical semantic change (Schlechtweg et al., 2020, 2021). We suggest to model WUGs with a Bayesian for- mulation of the Weighted Stochastic Block Model (WSBM), a generative model for random graphs popular in biology, physics and social sciences (Aicher et al., 2014; Peixoto, 2017). The basic assumption of WSBMs is that vertices belong to la- tent blocks (clusters), and that vertices in the same block are stochastically equivalent (i.e., they have edges drawn from the same distribution). Fitting the model is equivalent to determining the opti- mal latent block structure providing a clustering of word uses. By using a Bayesian probabilistic model of WUG data we aim to approach graded word mean- ing in a rigorous scientific way: We perform model selection, i.e., different models are compared ac- cording to their fit to the data, and the model which explains the data best is chosen as most adequate representation of the semantic structure behind human-annotated word uses. If blocks are equated with word senses, this allows us to approach this slippery and yet widely used concept in a novel way. We may test long-standing hypotheses such as whether a graded model allowing sense overlap is a better model than a discrete one (Kilgarriff, 1997; Erk et al., 2013; McCarthy et al., 2016). As a probabilistic model, the WSBM allows to …

Content truncated to 500 words