Hybrid Models for Lexical Acquisition of Correlated Styles

International Joint Conference on Natural Language Processing, pages 82–90, Nagoya, Japan, 14-18 October 2013. Hybrid Models for Lexical Acquisition of Correlated Styles Julian Brooke Department of Computer Science University of Toronto jbrooke@cs.toronto.edu Graeme Hirst Department of Computer Science University of Toronto gh@cs.toronto.edu Abstract Automated lexicon acquisition from cor- pora represents one way that large datasets can be leveraged to provide resources for a variety of NLP tasks. Our work applies techniques popularized in sentiment lexi- con acquisition and topic modeling to the broader task of creating a stylistic lexicon. A novel aspect of our approach is a fo- cus on multiple related styles, first extract- ing initial independent estimates of style based on co-occurrence with seeds in a large corpus, and then refining those es- timates based on the relationship between styles. We compare various promising implementation options, including vector space, Bayesian, and graph-based repre- sentations, and conclude that a hybrid ap- proach is indeed warranted. 1 Introduction Though lexical resources are useful for many NLP tasks, manual lexicon creation is often onerous, particularly for aspects of language for where full coverage requires hundred of thousands of annota- tions. This work deals with one such aspect which we refer to as stylistic variation. This should not be understood in a purely aesthetic sense, but as reflecting various high-level aspects of the text, in- cluding genre and social identity. Some tasks rele- vant to style so defined include genre classification (Kessler et al., 1997), author profiling (Rosenthal and McKeown, 2011), social relationship classi- fication (Peterson et al., 2011), sentiment anal- ysis (Wilson et al., 2005), readability classifica- tion (Collins-Thompson and Callan, 2005), and text generation (Hovy, 1990). Following the clas- sic work of Biber (1988), computational model- ing of style has often focused on textual statistics and the frequency of function words and syntac- tic categories. There are, of course, manually- constructed lists which capture some aspects of style, for instance resources related to psycholin- guistics (Coltheart, 1980), but these are necessar- ily limited in scope. Our interest is in provid- ing broad lexical coverage, potentially in any lan- guage. Here, we will show that style is particu- larly amenable to corpus-based automated lexical acquisition. Our approach to this problem is grounded in methods popularized for polarity lexicon creation (Turney and Littman, 2003), but we take a more holistic view than is typical, simultaneously tack- ling the acquisition of several styles in a single model. Not only is this theoretically warranted, due to the correlation effects resulting from the oral/literate spectrum of register, but we also show it can offer practical gains: our hybrid models first derive initial estimates of each style from a large social media corpus, and then refine these estimates based partially on the results from other styles. We demonstrate that various popular meth- ods are applicable to this problem, and indeed a single method might not provide the best results for all styles. For evaluation, we use a consensus annotation, the results of which also raise interest- …

Content truncated to 500 words