Near-synonym Lexical Choice in Latent Semantic Space

Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 1182–1190, Beijing, August 2010 Near-synonym Lexical Choice in Latent Semantic Space Tong Wang Department of Computer Science University of Toronto tong@cs.toronto.edu Graeme Hirst Department of Computer Science University of Toronto gh@cs.toronto.edu Abstract We explore the near-synonym lexical choice problem using a novel representa- tion of near-synonyms and their contexts in the latent semantic space. In contrast to traditional latent semantic analysis (LSA), our model is built on the lexical level of co-occurrence, which has been empir- ically proven to be effective in provid- ing higher dimensional information on the subtle differences among near-synonyms. By employing supervised learning on the latent features, our system achieves an ac- curacy of 74.5% in a “fill-in-the-blank” task. The improvement over the current state-of-the-art is statistically significant. We also formalize the notion of subtlety through its relation to semantic space di- mensionality. Using this formalization and our learning models, several of our intuitions about subtlety, dimensionality, and context are quantified and empirically tested. 1 Introduction Lexical choice is the process of selecting content words in language generation. Consciously or not, people encounter the task of lexical choice on a daily basis — when speaking, writing, and perhaps even in inner monologues. Its applica- tion also extends to various domains of natural language processing, including Natural Language Generation (NLG, Inkpen and Hirst 2006), writ- ers’ assistant systems (Inkpen, 2007), and sec- ond language (L2) teaching and learning (Ouyang et al., 2009). In the context of near-synonymy, the process of lexical choice becomes profoundly more com- plicated. This is partly because of the subtle nu- ances among near-synonyms, which can arguably differ along an infinite number of dimensions. Each dimension of variation carries differences in style, connotation, or even truth conditions into the discourse in question (Cruse, 1986), all mak- ing the seemingly intuitive problem of “choosing the right word for the right context” far from triv- ial even for native speakers of a language. In a widely-adopted “fill-in-the-blank” task, where the goal was to guess missing words (from a set of near-synonyms) in English sentences, two hu- man judges achieved an accuracy of about 80% (Inkpen, 2007). The current state-of-the-art accu- racy for an automated system is 69.9% (Islam and Inkpen, 2010). When the goal is to make plausible or even elegant lexical choices that best suit the con- text, the representation of that context becomes a key issue. We approach this problem in the la- tent semantic space, where transformed local co- occurrence data is capable of implicitly inducing global knowledge (Landauer and Dumais, 1997). A latent semantic space is constructed by reduc- ing the dimensionality of co-occurring linguistic units — typically words and documents as in La- tent Semantic Analysis (LSA). We refer to this level of association (LoA) as document LoA here- after. Although document LoA can benefit topical level classification (e.g., as in document retrieval, Deerwester et al. 1990), it is not necessarily suit- able for lexical-level tasks which might …

Content truncated to 500 words