Multilingual Content Moderation: A Case Study on Reddit

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3828–3844 May 2-6, 2023 ©2023 Association for Computational Linguistics Multilingual Content Moderation: A Case Study on Reddit Meng Ye1, Karan Sikka1, Katherine Atwell2, Sabit Hassan2 Ajay Divakaran1, Malihe Alikhani2 1Center for Vision Technologies, SRI International 2Department of Computer Science, University of Pittsburgh {first.last}@sri.com, {kaa139, sah259, malihe}@pitt.edu Abstract Content moderation is the process of flagging content based on pre-defined platform rules. There has been a growing need for AI moder- ators to safeguard users as well as protect the mental health of human moderators from trau- matic content. While prior works have focused on identifying hateful/offensive language, they are not adequate for meeting the challenges of content moderation since 1) moderation deci- sions are based on violation of rules, which subsumes detection of offensive speech, and 2) such rules often differ across communities which entails an adaptive solution. We propose to study the challenges of content moderation by introducing a multilingual dataset of 1.8 Million Reddit comments spanning 56 subred- dits in English, German, Spanish and French1. We perform extensive experimental analysis to highlight the underlying challenges and suggest related research problems such as cross-lingual transfer, learning under label noise (human bi- ases), transfer of moderation models, and pre- dicting the violated rule. Our dataset and anal- ysis can help better prepare for the challenges and opportunities of auto moderation. 1 Introduction Being able to moderate user-generated content is critical for online social media platforms. Sev- eral platforms employ human moderators to moni- tor user content to prevent the spread of misinfor- mation, adverse effects of hateful speech, fraud, etc (Geiger and Ribes, 2010; Dosono and Semaan, 2019; Wang et al., 2022; Jhaver et al., 2017). The moderators’ task is to remove improper content and/or suspend users posting such content. How- ever, reviewing and moderating each user com- ment is practically infeasible due to limited re- sources, especially during time-critical and large- scale events. More importantly, such modera- 1https://github.com/mye1225/multilingual_ content_mod.git “bien ah pap te felicito ..specs?” “jajajaj te re doli eh? xd” “… vaccine isn’t a perfectly safe treatment … the risk is so small that it is recommended for nearly everyone” “I am a physician. I'm just not verified on here. I'm a paediatric registrar in the UK. You're a paramedic right? …” “mein kanal zur aktuellen marktlage: wie du dein portfolio...” “mein kanal zur aktuellen marktlage: wie du dein portfolio...” “ok du sklaventreiber” “honein was wird dann aus meinen 24 dodgecoin” “… your muscles were likely not recovered and your nervous system ...” “nausea and dizziness are actually quite common side effects of the vaccine ...” “congrats mate! your story is very similar to mine …” “any one want to lose weight contact me via email = {{EMAIL}}. com” r/askdocs English Spanish German r/finanzen •No advertising •Provoke discussion r/bodyweightfitness r/argaming • Medical advices are allowed in r/askdocs, but not in r/bodyweightfitness. • An offensive comment that was not removed. • A removed comment …

Content truncated to 500 words