(Male, Bachelor) and (Female, Ph.D) have different connotations: Parallelly Annotated Stylistic Language Dataset with Multiple Personas

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1696–1706, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1696 (Male, Bachelor) and (Female, Ph.D) have different connotations: Parallelly Annotated Stylistic Language Dataset with Multiple Personas Dongyeop Kang Varun Gangal Eduard Hovy Carnegie Mellon University, Pittsburgh, PA, USA {dongyeok,vgangal,hovy}@cs.cmu.edu Abstract Stylistic variation in text needs to be stud- ied with different aspects including the writer’s personal traits, interpersonal relations, rhetoric, and more. Despite recent attempts on computational modeling of the variation, the lack of parallel corpora of style language makes it difficult to systematically control the stylistic change as well as evaluate such mod- els. We release PASTEL, the parallel and an- notated stylistic language dataset, that contains ≈ 41K parallel sentences (8.3K parallel sto- ries) annotated across different personas. Each persona has different styles in conjunction: gender, age, country, political view, education, ethnic, and time-of-writing. The dataset is col- lected from human annotators with solid con- trol of input denotation: not only preserving original meaning between text, but promot- ing stylistic diversity to annotators. We test the dataset on two interesting applications of style language, where PASTEL helps design appropriate experiment and evaluation. First, in predicting a target style (e.g., male or fe- male in gender) given a text, multiple styles of PASTEL make other external style variables controlled (or fixed), which is a more accu- rate experimental design. Second, a simple supervised model with our parallel text out- performs the unsupervised models using non- parallel text in style transfer. Our dataset is publicly available1. 1 Introduction Hovy (1987) claims that appropriately varying the style of text often conveys more information than is contained in the literal meaning of the words. He defines the roles of styles in text variation by pragmatics aspects (e.g., relationship between them) and rhetorical goals (e.g., formality), and provides example texts of how they are tightly 1https://github.com/dykang/PASTEL coupled in practice. Similarly, Biber (1991) cat- egorizes components of conversational situation by participants’ characteristics such as their roles, personal characteristics, and group characteristics (e.g., social class). Despite the broad definition of style, this work mainly focuses on one specific aspect of style, pragmatics aspects in group char- acteristics of speakers, which is also called per- sona. Particularly, we look at multiple types of group characteristics in conjunction, such as gen- der, age, education level, and more. Stylistic variation in text primarily manifest themselves at the different levels of textual fea- tures: lexical features (e.g., word choice), syntac- tic features (e.g., preference for the passive voice) and even pragmatics, while preserving the original meaning of given text (DiMarco and Hirst, 1990). Connecting such textual features to someone’s persona is an important study to understand stylis- tic variation of language. For example, do highly educated people write longer sentences (Bloom- field, 1927)? Are Hispanic and East Asian people more likely to drop pronouns (White, 1985)? Are elder people likely to use lesser …

Content truncated to 500 words