Discovering Sociolinguistic Associations with Structured Sparsity

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 1365–1374, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Discovering Sociolinguistic Associations with Structured Sparsity Jacob Eisenstein Noah A. Smith School of Computer ScienceEric P. Xing Carnegie Mellon University Pittsburgh, PA 15213, USA {jacobeis,nasmith,epxing}@cs.cmu.edu Abstract We present a method to discover robust and interpretable sociolinguistic associations from raw geotagged text data. Using aggregate de- mographic statistics about the authors’ geo- graphic communities, we solve a multi-output regression problem between demographics and lexical frequencies. By imposing a com- posite ℓ1,∞ regularizer, we obtain structured sparsity, driving entire rows of coefficients to zero. We perform two regression studies. First, we use term frequencies to predict de- mographic attributes; our method identifies a compact set of words that are strongly asso- ciated with author demographics. Next, we conjoin demographic attributes into features, which we use to predict term frequencies. The composite regularizer identifies a small num- ber of features, which correspond to com- munities of authors united by shared demo- graphic and linguistic properties. 1 Introduction How is language influenced by the speaker’s so- ciocultural identity? Quantitative sociolinguistics usually addresses this question through carefully crafted studies that correlate individual demographic attributes and linguistic variables—for example, the interaction between income and the “dropped r” fea- ture of the New York accent (Labov, 1966). But such studies require the knowledge to select the “dropped r” and the speaker’s income, from thou- sands of other possibilities. In this paper, we present a method to acquire such patterns from raw data. Us- ing multi-output regression with structured sparsity, our method identifies a small subset of lexical items that are most influenced by demographics, and dis- covers conjunctions of demographic attributes that are especially salient for lexical variation. Sociolinguistic associations are difficult to model, because the space of potentially relevant interactions is large and complex. On the linguistic side there are thousands of possible variables, even if we limit ourselves to unigram lexical features. On the demo- graphic side, the interaction between demographic attributes is often non-linear: for example, gender may negate or amplify class-based language differ- ences (Zhang, 2005). Thus, additive models which assume that each demographic attribute makes a lin- ear contribution are inadequate. In this paper, we explore the large space of po- tential sociolinguistic associations using structured sparsity. We treat the relationship between language and demographics as a set of multi-input, multi- output regression problems. The regression coeffi- cients are arranged in a matrix, with rows indicating predictors and columns indicating outputs. We ap- ply a composite regularizer that drives entire rows of the coefficient matrix to zero, yielding compact, interpretable models that reuse features across dif- ferent outputs. If we treat the lexical frequencies as inputs and the author’s demographics as outputs, the induced sparsity pattern reveals the set of lexi- cal items that is most closely tied to demographics. If we treat the demographic attributes as inputs and build a model to predict the text, we can …

Content truncated to 500 words