SWEAT: Scoring Polarization of Topics across Different Corpora

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10065–10072 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 10065 SWEAT: Scoring Polarization of Topics across Different Corpora Federico Bianchi Bocconi University Via Sarfatti 25 Milan, Italy f.bianchi@unibocconi.it Marco Marelli University of Milano-Bicocca Piazza dell’Ateneo Nuovo, 1 Milan, Italy marco.marelli@unimib.it Paolo Nicoli University of Milano-Bicocca Viale Sarca 336 Milan, Italy p.nicoli@campus.unimib.it Matteo Palmonari University of Milano-Bicocca Viale Sarca 336 Milan, Italy matteo.palmonari@unimib.it Abstract Understanding differences of viewpoints across corpora is a fundamental task for computational social sciences. In this paper, we propose the Sliced Word Embedding Association Test (SWEAT), a novel statistical measure to compute the relative polarization of a topical wordset across two distributional representations. To this end, SWEAT uses two additional wordsets, deemed to have opposite valence, to represent two different poles. We validate our approach and illustrate a case study to show the usefulness of the introduced measure. 1 Introduction In this short paper, we introduce a method to score polarization of different corpora with respect to a given topic. The method is intended to support studies where two different corpora are compared (e.g., news sources inspired by different political positions or social communities characterized by different viewpoints) to investigate whether they convey implicit attitudes towards a given topic. This corpus-wise comparison - its main peculiarity with respect to the large body of work proposed to study and correct bias in NLP models (we re- fer to (Garrido-Muñoz et al., 2021) for a detailed survey on bias in NLP models) - is based on a new measure that we introduce, the Sliced Word Embedding Association Test (SWEAT). SWEAT is an extension of the Word Embedding Association Test (WEAT) proposed by Caliskan et al. (2017), which measures the comparative po- larization for a pair of topical wordsets (e.g., insects and flowers) against a pair of attribute wordsets (e.g., pleasant and unpleasant) in a single-corpus distributional model (e.g. 1950 American newspa- per articles). In this context, with polarization we refer to the phenomenon for which two communi- ties have opposite attitudes against some topic. With SWEAT we extend this approach by mea- suring the relative polarization for a single topical wordset - the topic, using a pair of stable attribute wordsets deemed to have opposite valence - the two poles, in a pair of aligned distributional models representing the semantics of two different corpora. We explain the rationale behind SWEAT with an ex- ample. Suppose that we want to investigate whether two different Italian news sources, e.g., La Repub- blica (known to be closer to center-left political positions) and Il Giornale (known to be closer to center-right political positions) hold different and opposite viewpoints about a topic, e.g., “Berlus- coni” (a reference center-right Italian politician in the recent past). We can collect a news corpus from La Repubblica and one from Il Giornale to train two different distributional models in such a way that they are aligned (Hamilton et al., 2016b; Carlo et al., 2019; Cassani …

Content truncated to 500 words