Constructing Semantic Space Models from Parsed Corpora

Constructing Semantic Space Models from Parsed Corpora Sebastian Padó Department of Computational Linguistics Saarland University PO Box 15 11 50 66041 Saarbrücken, Germany pado@coli.uni-sb.de Mirella Lapata Department of Computer Science University of Sheffield Regent Court, 211 Portobello Street Sheffield S1 4DP, UK mlap@dcs.shef.ac.uk Abstract Traditional vector-based models use word co-occurrence counts from large corpora to represent lexical meaning. In this pa- per we present a novel approach for con- structing semantic spaces that takes syn- tactic relations into account. We introduce a formalisation for this class of models and evaluate their adequacy on two mod- elling tasks: semantic priming and auto- matic discrimination of lexical relations. 1 Introduction Vector-based models of word co-occurrence have proved a useful representational framework for a variety of natural language processing (NLP) tasks such as word sense discrimination (Schütze, 1998), text segmentation (Choi et al., 2001), contextual spelling correction (Jones and Martin, 1997), auto- matic thesaurus extraction (Grefenstette, 1994), and notably information retrieval (Salton et al., 1975). Vector-based representations of lexical meaning have been also popular in cognitive science and fi gure prominently in a variety of modelling stud- ies ranging from similarity judgements (McDonald, 2000) to semantic priming (Lund and Burgess, 1996; Lowe and McDonald, 2000) and text comprehension (Landauer and Dumais, 1997). In this approach semantic information is extracted from large bodies of text under the assumption that the context surrounding a given word provides im- portant information about its meaning. The semantic properties of words are represented by vectors that are constructed from the observed distributional pat- terns of co-occurrence of their neighbouring words. Co-occurrence information is typically collected in a frequency matrix, where each row corresponds to a unique target word and each column represents its linguistic context. Contexts are defi ned as a small number of words surrounding the target word (Lund and Burgess, 1996; Lowe and McDonald, 2000) or as entire para- graphs, even documents (Landauer and Dumais, 1997). Context is typically treated as a set of unordered words, although in some cases syntac- tic information is taken into account (Lin, 1998; Grefenstette, 1994; Lee, 1999). A word can be thus viewed as a point in an n-dimensional semantic space. The semantic similarity between words can be then mathematically computed by measuring the distance between points in the semantic space using a metric such as cosine or Euclidean distance. In the variants of vector-based models where no linguistic knowledge is used, differences among parts of speech for the same word (e.g., to drink vs. a drink) are not taken into account in the con- struction of the semantic space, although in some cases word lexemes are used rather than word sur- face forms (Lowe and McDonald, 2000; McDonald, 2000). Minimal assumptions are made with respect to syntactic dependencies among words. In fact it is assumed that all context words within a certain dis- tance from the target word are semantically relevant. The lack of syntactic information makes the build- ing of semantic space models relatively straightfor- ward and language …

Content truncated to 500 words