A Latent Variable Model for Geographic Lexical Variation

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1277–1287, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics A Latent Variable Model for Geographic Lexical Variation Jacob Eisenstein Brendan O’Connor Noah A. Smith Eric P. Xing School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213, USA {jacobeis,brendano,nasmith,epxing}@cs.cmu.edu Abstract The rapid growth of geotagged social media raises new computational possibilities for in- vestigating geographic linguistic variation. In this paper, we present a multi-level generative model that reasons jointly about latent topics and geographical regions. High-level topics such as “sports” or “entertainment” are ren- dered differently in each geographic region, revealing topic-specific regional distinctions. Applied to a new dataset of geotagged mi- croblogs, our model recovers coherent top- ics and their regional variants, while identi- fying geographic areas of linguistic consis- tency. The model also enables prediction of an author’s geographic location from raw text, outperforming both text regression and super- vised topic models. 1 Introduction Sociolinguistics and dialectology study how lan- guage varies across social and regional contexts. Quantitative research in these fields generally pro- ceeds by counting the frequency of a handful of previously-identified linguistic variables: pairs of phonological, lexical, or morphosyntactic features that are semantically equivalent, but whose fre- quency depends on social, geographical, or other factors (Paolillo, 2002; Chambers, 2009). It is left to the experimenter to determine which variables will be considered, and there is no obvious procedure for drawing inferences from the distribution of multiple variables. In this paper, we present a method for identifying geographically-aligned lexical variation directly from raw text. Our approach takes the form of a probabilistic graphical model capable of iden- tifying both geographically-salient terms and coher- ent linguistic communities. One challenge in the study of lexical variation is that term frequencies are influenced by a variety of factors, such as the topic of discourse. We address this issue by adding latent variables that allow us to model topical variation explicitly. We hypothesize that geography and topic interact, as “pure” topi- cal lexical distributions are corrupted by geographi- cal factors; for example, a sports-related topic will be rendered differently in New York and Califor- nia. Each author is imbued with a latent “region” indicator, which both selects the regional variant of each topic, and generates the author’s observed ge- ographical location. The regional corruption of top- ics is modeled through a cascade of logistic normal priors—a general modeling approach which we call cascading topic models. The resulting system has multiple capabilities, including: (i) analyzing lexi- cal variation by both topic and geography; (ii) seg- menting geographical space into coherent linguistic communities; (iii) predicting author location based on text alone. This research is only possible due to the rapid growth of social media. Our dataset is derived from the microblogging website Twitter,1 which permits users to post short messages to the public. Many users of Twitter also supply exact geographical co- ordinates from GPS-enabled devices (e.g., mobile phones),2 yielding geotagged text data. Text in computer-mediated …

Content truncated to 500 words