UMUTextStats: A linguistic feature extraction tool for Spanish

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6035–6044 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6035 UMUTextStats: A linguistic feature extraction tool for Spanish Jos´e Antonio Garc´ıa-D´ıaz, Pedro Jos´e Vivancos-Vicente, ´Angela Almela, Rafael Valencia-Garc´ıa Facultad de Inform´atica, Universidad de Murcia, Campus de Espinardo, 30100, Spain V´ocali Sistemas Inteligentes S.L., Parque Cient´ıfico de Murcia, Complejo de Espinardo, 30100, Spain Facultad de Letras, Universidad de Murcia, Campus de La Merced, 30001, Murcia, Spain {joseantonio.garcia8, angelalm, valencia}@um.es pedro.vivancos@vocali.net Abstract Feature Engineering consists in the application of domain knowledge to select and transform relevant features to build efficient machine learning models. In the Natural Language Processing field, the state of the art concerning automatic document classification tasks relies on word and sentence embeddings built upon deep learning models based on transformers that have outperformed the competition in several tasks. However, the models built from these embeddings are usually difficult to interpret. On the contrary, linguistic features are easy to understand, they result in simpler models, and they usually achieve encouraging results. Moreover, both linguistic features and embeddings can be combined with different strategies which result in more reliable machine-learning models. The de facto tool for extracting linguistic features in Spanish is LIWC. However, this software does not consider specific linguistic phenomena of Spanish such as grammatical gender and lacks certain verb tenses. In order to solve these drawbacks, we have developed UMUTextStats, a linguistic extraction tool designed from scratch for Spanish. Furthermore, this tool has been validated to conduct different experiments in areas such as infodemiology, hate-speech detection, author profiling, authorship verification, humour or irony detection, among others. The results indicate that the combination of linguistic features and embeddings based on transformers are beneficial in automatic document classification. Keywords: Linguistic Features, Feature Engineering, Natural Language Processing 1. Introduction Online communication has opened a new way of ex- ploring how people communicate and interact with each other. Recent trends in Natural Language Process- ing, Artificial Intelligence and Information Retrieval have eased the analysis of large amounts of data that can be used for conducting automatic document clas- sification tasks, such as Sentiment Analysis (SA), with applications in marketing in order to gain faster insights about brands, products and services. Other popular applications of automatic document classification are authorship attribution (AA), which can help uncover anonymous threats, plagiarism detection, and cyberbul- lying and hate-speech (HS) detection, building safer so- cial environments. The state of the art concerning automatic document classification is focused on pre-trained embeddings that have been learned with large corpora. These mod- els learn to represent words as vectors of fixed size and arrange them within the latent space, clustering words that have a close relationship. These models are learned based on unsupervised tasks, such as Masked Language Modelling (MLM). Moreover, transformers and attention mechanisms have enabled the learning of contextual embeddings, in which words are aware of surrounding ones, solving problems partially related to polysemy and word disambiguation. Word embeddings can …

Content truncated to 500 words