Controversy and Conformity: from Generalized to Personalized Aggressiveness Detection

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 5915–5926 August 1–6, 2021. ©2021 Association for Computational Linguistics 5915 Controversy and Conformity: from Generalized to Personalized Aggressiveness Detection Kamil Kanclerz1, Alicja Figas2, Marcin Gruza1, Tomasz Kajdanowicz1, Jan Koco´n1, Daria Puchalska2, Przemysław Kazienko1 Wrocław University of Science and Technology 27 Wybrze˙ze Wyspia´nskiego st. Wrocław, Poland 1{kamil.kanclerz,marcin.gruza,tomasz.kajdanowicz,jan.kocon, kazienko}@pwr.edu.pl 2{238442,234800}@student.pwr.edu.pl Abstract There is content such as hate speech, offen- sive, toxic or aggressive documents, which are perceived differently by their consumers. They are commonly identified using classifiers solely based on textual content that general- ize pre-agreed meanings of difficult problems. Such models provide the same results for each user, which leads to high misclassification rate observable especially for contentious, aggres- sive documents. Both document controversy and user nonconformity require new solutions. Therefore, we propose novel personalized ap- proaches that respect individual beliefs ex- pressed by either user conformity-based mea- sures or various embeddings of their previous text annotations. We found that only a few annotations of most controversial documents are enough for all our personalization meth- ods to significantly outperform classic, gener- alized solutions. The more controversial the content, the greater the gain. The personalized solutions may be used to efficiently filter un- wanted aggressive content in the way adjusted to a given person. 1 Introduction Unfortunately, in the pursuit of knowledge on the Internet, one may come across content that they consider inappropriate for various reasons, such as being too aggressive. Many users notoriously come across content that offends them while surfing the Internet. This can cause discomfort and discourage from further expansion of knowledge. To avoid this, it is important to effectively filter out content that a given user may find unwanted. This poses a risk of erroneous assessment of whether a given text is con- sidered inappropriate by a given person. For that purpose, we need to extend commonly applied gen- eralizing solutions and develop personalized meth- ods that take into account beliefs and preferences of the individual user. We expect this information can be obtained from the individual’s prior opin- ions about the offensiveness of some texts. Then, it is crucial to select the relevant texts that allow deriving as much information about users prefer- ences as possible. Our new idea is to use some known, most controversial texts whose offensive- ness is very ambiguous and depends more on sub- jective personal judgment. We examined how many documents has to be annotated by a given user to encapsulate their beliefs sufficiently and to improve personalized reasoning. Independently, we consid- ered personal measures quantifying conformity of each individual. In other words, we measured to what extent a person evaluates documents simi- larly to others, i.e. "is a part of the mainstream". The conformity measures are used as input fea- tures for the classifier. This way, it is possible to find out the user beliefs based on their opinions regarding a relatively small number of texts. In …

Content truncated to 500 words