Detecting Text Formality: A Study of Text Classification Approaches

Proceedings of Recent Advances in Natural Language Processing, pages 274–284 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_031 274 Detecting Text Formality: A Study of Text Classification Approaches Daryna Dementieva1, Nikolay Babakov2, and Alexander Panchenko3,4 1Technical University of Munich 2Centro Singular de Investigaci´on en Tecnolox´ıas Intelixentes (CiTIUS), Universidade de Santiago de Compostela 3Skolkovo Institute of Science and Technology, 4Artificial Intelligence Research Institute daryna.dementieva@tum.de, nikolay.babakov@usc.es, a.panchenko@skol.tech Abstract Formality is one of the important characteris- tics of text documents. The automatic detection of the formality level of a text is potentially beneficial for various natural language process- ing tasks. Before, two large-scale datasets were introduced for multiple languages fea- turing formality annotation—GYAFC and X- FORMAL. However, they were primarily used for the training of style transfer models. At the same time, the detection of text formal- ity on its own may also be a useful appli- cation. This work proposes the first to our knowledge systematic study of formality de- tection methods based on statistical, neural- based, and Transformer-based machine learn- ing methods and delivers the best-performing models for public usage. We conducted three types of experiments – monolingual, multi- lingual, and cross-lingual. The study shows the overcome of Char BiLSTM model over Transformer-based ones for the monolingual and multilingual formality classification task, while Transformer-based classifiers are more stable to cross-lingual knowledge transfer. 1 Introduction According to Joos (1976), five different types of text formality are commonly identified in Linguis- tics: frozen style, formal style, consultative style, casual style, and intimate style. The correct use of style is important for fluent human communica- tion and, therefore, for fluent human-to-machine communication and various Natural Language Pro- cessing (NLP) systems. The examples of formal and informal samples for English, Brazilian Portuguese, French, and Ital- ian languages are provided in Table 1. As we can see, for informal sentences, several attributes are typical – the usage of spoken abbreviations (for instance, lol), non-standard capitalization of words (all words are written in upper case), and lack of punctuation. On the contrary, in formal samples, all necessary punctuation is present, standard capi- talization is used, some opening expressions can be observed in sentences (for example, in my opinion). These examples are taken from two only cur- rently available text collections with formality an- notation are GYAFC (Rao and Tetreault, 2018) and X-FORMAL (Briakou et al., 2021). However, these datasets were primarily introduced for the task of style transfer. In this paper, we propose to look at these data sets from a different angle. Even for the evaluation of the results of formality style trans- fer, we need to calculate style transfer accuracy. While there is ongoing work of developing auto- matic evaluation metrics for formality style transfer in general (Lai et al., 2022), this work introduces a systematic evaluation of formality style classifiers. In this paper, we aim at closing the gap by proposing a comprehensive computational study of various text categorization approaches. Namely, we argue that NLP practitioners will be benefiting from the knowledge of answers to the following …

Content truncated to 500 words