Predicting Word Clipping with Latent Semantic Analysis

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 1392–1396, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Predicting Word Clipping with Latent Semantic Analysis Julian Brooke Tong Wang Graeme Hirst Department of Computer Science University of Toronto {jbrooke,tong,gh}@cs.toronto.edu Abstract In this paper, we compare a resource- driven approach with a task-specific clas- sification model for a new near-synonym word choice sub-task, predicting whether a full or a clipped form of a word will be used (e.g. doctor or doc) in a given con- text. Our results indicate that the resource- driven approach, the use of a formality lexicon, can provide competitive perfor- mance, with the parameters of the task- specific model mirroring the parameters under which the lexicon was built. 1 Introduction Lexical resources, though the focus of much work in computational linguistics, often compare poorly to direct statistical methods when applied to prob- lems such as sentiment classification (Kennedy and Inkpen, 2006). Nevertheless, such resources offer advantages in terms of human interpretability and portability to many different tasks (Argamon et al., 2007). In this paper, we introduce a new sub-task of near-synonym word choice (Edmonds and Hirst, 2002), prediction of word clipping, in order to provide some new evidence that resource- driven approaches have general potential. Clipping is a type of word formation where the beginning and/or the end of a longer word is omitted (Kreidler, 1979). This phenomenon is attested in various languages; well-known exam- ples in English include words such as hippo (hip- popotamus) and blog (weblog). Clipping and re- lated kinds of word formation have received at- tention in computational linguistics with respect to the task of identifying source words from ab- breviated forms, which has been studied, for in- stance, in the biomedical and text messaging do- mains (Okazaki and Ananiadou, 2006; Cook and Stevenson, 2009). Compared to many near-synonyms, clipped forms have the important property that the dif- ferences between full and abbreviated forms are almost entirely connotational or stylistic, closely tied to the formality of the discourse.1 This fact allows us to pursue two distinct though related approaches to this task, comparing a supervised model of word choice (Wang and Hirst, 2010) with a mostly unsupervised system that leverages an automatically-built lexicon of formality (Brooke et al., 2010). Our findings indicate that the lexicon-based method is highly competitive with the supervised, task-specific method. Both mod- els approach the human performance evidenced in an independent crowdsourced annotation. 2 Methods 2.1 Latent Semantic Analysis Both approaches that we are investigating make use of Latent Semantic Analysis (LSA) as a dimensionality-reduction technique (Landauer and Dumais, 1997).2 In LSA, the first step is to create a matrix representing the association be- tween words as determined by their co-occurrence in a corpus, and then apply singular value decom- position (SVD) to identify the first k most signifi- cant dimensions of variation. After this step, each word can be represented as a vector of length k, which can be compared or combined with …

Content truncated to 500 words