A Multi-Dimensional Bayesian Approach to Lexical Style

Proceedings of NAACL-HLT 2013, pages 673–679, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics A Multi-Dimensional Bayesian Approach to Lexical Style Julian Brooke Department of Computer Science University of Toronto jbrooke@cs.toronto.edu Graeme Hirst Department of Computer Science University of Toronto gh@cs.toronto.edu Abstract We adapt the popular LDA topic model (Blei et al., 2003) to the representation of stylistic lexical information, evaluating our model on the basis of human-interpretability at the word and text level. We show, in particular, that this model can be applied to the task of inducing stylistic lexicons, and that a multi-dimensional approach is warranted given the correlations among stylistic dimensions. 1 Introduction In language, stylistic variation is a reflection of var- ious contextual factors, including the backgrounds of and relationship between the parties involved. Although in the context of prescriptive linguistics (Strunk and White, 1979), style is often assumed to be a matter of aesthetics, the stylistic intuitions of language users are inextricably linked to the conven- tions of register and genre (Biber and Conrad, 2009). Intentional or not, stylistic differences play a role in numerous NLP tasks. Examples include genre classification (Kessler et al., 1997), author profil- ing (Garera and Yarowsky, 2009; Rosenthal and Mc- Keown, 2011), social relationship classification (Pe- terson et al., 2011), sentiment analysis (Wilson et al., 2005), readability classification (Collins-Thompson and Callan, 2005), and text generation (Hovy, 1990; Inkpen and Hirst, 2006). Following the classic work of Biber (1988), computational modeling of style has often focused on textual statistics and the fre- quency of function words and syntactic categories. When content words are considered, they are of- ten limited to manually-constructed lists (Argamon et al., 2007), or used as individual features for su- pervised classification, which can be confounded by topic (Petrenz and Webber, 2011) or fail in the face of lexical variety. Our interest is models that offer broad lexical coverage of human-identifiable stylis- tic variation. Research most similar to ours has focused on clas- sifying the lexicon in terms of individual aspects rel- evant to style (e.g. formality, specificity, readability) (Brooke et al., 2010; Pan and Yang, 2010; Kidwell et al., 2009) and a large body of research on the in- duction of polarity lexicons, in particular from large corpora (Turney, 2002; Kaji and Kitsuregawa, 2007; Velikovich et al., 2010). Our work is the first to rep- resent multiple dimensions of style in a single statis- tical model, adapting latent Dirichlet allocation (Blei et al., 2003), a Bayesian ‘topic’ model, to our stylis- tic purposes; as such, our approach also follows on recent interest in the interpretability of topic-model topics (Chang et al., 2009; Newman et al., 2011). We show that our model can be used for acquisition of stylistic lexicons, and we also evaluate the model relative to theories of register variation and the ex- pected stylistic character of particular genres. 2 Model 2.1 Linguistic foundations In English manuals of style and other prescriptivist texts (Fowler and Fowler, 1906; Gunning, 1952; Follett, 1966; Strunk and White, …

Content truncated to 500 words