Women’s Syntactic Resilience and Men’s Grammatical Luck: Gender-Bias in Part-of-Speech Tagging and Dependency Parsing

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3493–3498 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3493 Women’s Syntactic Resilience and Men’s Grammatical Luck: Gender-Bias in Part-of-Speech Tagging and Dependency Parsing Aparna Garimella1, Carmen Banea1, Dirk Hovy2, Rada Mihalcea1 1Computer Science and Engineering, University of Michigan, Ann Arbor, MI {gaparna,carmennb, mihalcea}@umich.edu 2Department of Marketing, Bocconi University, Milan, Italy dirk.hovy@unibocconi.it Abstract Several linguistic studies have shown the prevalence of various lexical and grammatical patterns in texts authored by a person of a par- ticular gender, but models for part-of-speech tagging and dependency parsing have still not adapted to account for these differences. To address this, we annotate the Wall Street Jour- nal part of the Penn Treebank with the gender information of the articles’ authors, and build taggers and parsers trained on this data that show performance differences in text written by men and women. Further analyses reveal numerous part-of-speech tags and syntactic re- lations whose prediction performances bene- fit from the prevalence of a specific gender in the training data. The results underscore the importance of accounting for gendered dif- ferences in syntactic tasks, and outline future venues for developing more accurate taggers and parsers. We release our data to the re- search community. 1 Introduction Sociolinguistic studies have shown that people use grammatical features to signal the speakers’ membership in a demographic group, with a fo- cus on gender (Vigliocco and Franck, 1999; Mon- dorf, 2002; Eckert and McConnell-Ginet, 2013). Mondorf (2002) shows systemic differences in the usage of various types of clauses and their po- sitions for men and women, stating that women have a higher usage of adverbial (accordingly, consequently1), causal (since, because), condi- tional (if, when) and purpose (so, in order that) clauses, while men tend to use more concessive clauses (but, although, whereas). Similar results hold across various languages in Johannsen et al. (2015). 1We exemplify in parentheses conjunctions or conjunctive adverbs that introduce and link in a subordinating relationship the given type of subordinate clause. This correlation between grammatical features and gender has important ramifications for statis- tical models of syntax: if the training sample is unbalanced, these differences inadvertently intro- duce a strong gender bias into the training data. Such demographic imbalances are amplified by the model (Zhao et al., 2017), which in turn can be detrimental to members of the underrepresented demographic groups (Jørgensen et al., 2015; Hovy and Søgaard, 2015; Hovy and Spruit, 2016). Since several works use syntactic analysis to improve tasks ranging from data-driven dependency pars- ing (Gadde et al., 2010) to sentiment classification (Moilanen and Pulman, 2007; Socher et al., 2013), underlying model biases end up affecting the per- formance of a wide range of applications. While data bias can be overcome by accounting for de- mographics, and can even improve classification performance (Volkova et al., 2013; Hovy, 2015; Bolukbasi et al., 2016; Benton et al., 2017; Zhao et al., 2017; Lynn et al., 2017), there is …

Content truncated to 500 words