Stigma Annotation Scheme and Stigmatized Language Detection in Health-Care Discussions on Social Media

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 1178–1190 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 1178 Stigma Annotation Scheme and Stigmatized Language Detection in Health-Care Discussions on Social Media Nadiya Straton, Hyeju Jang, Raymond Ng Center for Business Data Analytics, Data Science Institute, Data Science Institute Copenhagen Business School, University of British Columbia, University of British Columbia ns.digi@cbs.dk, hyejuj@cs.ubc.ca, rng@cs.ubc.ca Abstract Much research has been done within the social sciences on the interpretation and influence of stigma on human behaviour and health, which result in out-of-group exclusion, distancing, cognitive separation, status loss, discrimination, in-group pressure, and often lead to disengagement, non-adherence to treatment plan, and prescriptions by the doctor. However, little work has been conducted on computational identification of stigma in general and in social media discourse in particular. In this paper, we develop the annotation scheme and improve the annotation process for stigma identification, which can be applied to other health-care domains. The data from pro-vaccination and anti-vaccination discussion groups are annotated by trained annotators who have professional background in social science and health-care studies, therefore the group can be considered experts on the subject in comparison to non-expert crowd. Amazon MTurk annotators is another group of annotator with no knowledge on their education background, they are initially treated as non-expert crowd on the subject matter of stigma. We analyze the annotations with visualisation techniques, features from LIWC (Linguistic Inquiry and Word Count) list and make prediction based on bi-grams with traditional and deep learning models. Data augmentation method and application of CNN show high performance accuracy in comparison to other models. Success of the rigorous annotation process on identifying stigma is reconfirmed by achieving high prediction rate with CNN. Keywords: Annotation process, Stigma Annotation Scheme, Social media, CNN, N-grams. 1. Introduction In this paper, we study stigma – we aim to distinguish stigmatised language from non-stigmatised using machine learning and natural language processing (NLP). We for- mulate the problem as text classification, and explore dif- ferent classification algorithms in order to understand the components of “health stigma” better. Stigma is a concept related to bias, prejudice, and stereo- type. According to the definition in Merriam-Webster (2011), stigma is a mark of shame or discredit, while stereo- type is a standardized mental picture that is held in com- mon by members of a group and that represents an oversim- plified opinion, prejudiced attitude, or uncritical judgment. Bias a personal and unreasoned judgment, and prejudice is preconceived opinion, formed without just grounds or be- fore sufficient knowledge, an irrational attitude of hostility directed against an individual, a group, a race, or their sup- posed characteristics. Although these concepts are slightly different, they are all closely related in that all these neg- ative attitudes can lead to discrimination. In the current study, we will use stigma, bias, prejudice, and stereotype interchangeably. Stigma is a barrier to quality healthcare, which often un- dermines health seeking behaviors like diagnosis and en- …

Content truncated to 500 words