Olá, Bonjour, Salve! XFORMAL: A Benchmark for Multilingual Formality Style Transfer

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3199–3216 June 6–11, 2021. ©2021 Association for Computational Linguistics 3199 Olá, Bonjour, Salve! XFORMAL: A Benchmark for Multilingual Formality Style Transfer Eleftheria Briakou∗ University of Maryland ebriakou@cs.umd.edu Di Lu Dataminr, Inc. dlu@dataminr.com Ke Zhang Dataminr, Inc. kzhang@dataminr.com Joel Tetreault Dataminr, Inc. jtetreault@dataminr.com Abstract We take the first step towards multilingual style transfer by creating and releasing XFOR- MAL, a benchmark of multiple formal refor- mulations of informal text in Brazilian Por- tuguese, French, and Italian. Results on XFOR- MAL suggest that state-of-the-art style transfer approaches perform close to simple baselines, indicating that style transfer is even more chal- lenging when moving multilingual.1 1 Introduction Style Transfer (ST) is the task of automatically transforming text in one style into another (for ex- ample, making an impolite request more polite). Most work in this growing field has focused primar- ily on style transfer within English, while covering different languages has received disproportional interest. Concretely, out of 35 ST papers we re- viewed, all of them report results for ST within English text, while there is just a single work cov- ering each of the following languages: Chinese, Russian, Latvian, Estonian, and French (Shang et al., 2019; Tikhonov et al., 2019; Korotkova et al., 2019; Niu et al., 2018). Notably, even though some efforts have been made towards multilingual ST, researchers are limited to providing system outputs as a means of evaluation, and progress is hampered by the scarcity of resources for most languages. At the same time, ST lies at the core of human com- munication: when humans produce language, they condition their choice of grammatical and lexical transformations to a target audience and a specific situation. Among the many possible stylistic varia- tions, Heylighen et al. (1999) argue that “a dimen- sion similar to formality appears as the most im- portant and universal feature distinguishing styles, registers or genres in different languages”. Con- sider the informal excerpts and their formal refor- mulations in French (FR) and Brazilian Portuguese ∗Work done as a Research Intern at Dataminr, Inc. 1Code and data: https://github.com/Elbria/xformal-FoST BRAZILIAN-PORTUGUESE saiam disso, força de vontade!! get out of it, willpower!! Abandonem essa situação, tenham força de vontade. Abandon this situation, have willpower! FRENCH Il avait les yeux braqués ailleurs. He had his eyes fixed elsewhere. Il ne prêtait pas attention à la situation. He was not paying attention to the situation. ITALIAN in bocca al lupo! good luck! Ti rivolgo un sincero augurio! I send you a sincere wish! Table 1: Informal-Formal pairs in XFORMAL. (BR-PT) in Table 1. Both informal-formal pairs share the same content. However, the informal language conveys more information than is con- tained in the literal meaning of the words (Hovy, 1987). These examples relate to the notion of deep formality (Heylighen et al., 1999), where the ulti- mate goal is that of adding the context needed to disambiguate an expression. On the other hand, …

Content truncated to 500 words