Overview of the 2020 ALTA Shared Task: Assess Human Behaviour

Overview of the 2020 ALTA Shared Task: Assess Human Behaviour Diego Moll´a Department of Computing Macquarie University diego.molla-aliod@mq.edu.au Abstract The 2020 ALTA shared task is the 11th in- stance of a series of shared tasks organised by ALTA since 2010. The task is to classify texts posted in social media according to hu- man judgements expressed in them. The data used for this task is a subset of SemEval 2018 AIT DISC, which has been annotated by do- main experts for this task. In this paper we in- troduce the task, describe the data and present the results of participating systems. 1 Introduction Human behaviour can be negatively or positively assessed based on a reference set of social norms. When judgement is explicitly stated in narratives, e.g., “They are hard-working and honest.”, we can attempt to encounter appraisal words such as “hard- working” and “honest” used between interlocutors for advancing their judgement. Attitude positioning plays an important role in Martin and White’s (2005) Appraisal framework1 (AF) for analysing someone’s use of evaluative language to negotiate solidarity. To the best of our knowledge, no prior work has attempted to automatically codify text using the AF judgement categories. The goal of the 2020 ALTA shared task is to develop a computational model that can identify and classify judgements expressed in textual segments. Participants are challenged to predict the judgement appraised by classifying each short-text message into one or more label candidates (or none): normality, capacity, tenacity, veracity, propriety. 2 The 2020 ALTA Shared Task The 2020 ALTA Shared Task is the 11th of the shared tasks organised by the Australasian Lan- 1https://www.grammatics.com/appraisal/ guage Technology Association (ALTA). As in pre- vious shared tasks, it targets university students with programming experience, but it is also open to graduates and professionals. The general objec- tive of these shared tasks is to introduce interested people to the sort of problems that are the subject of active research in a field of natural language processing. Depending on the availability of data, the tasks have ranged from classic but challenging tasks to tasks linked to very hot topics of research. Details of the 2020 ALTA Shared task and past tasks can be found in the 2020 ALTA Shared Task website.2 There are no limitations on the size of the teams or the means that they may use to solve the problem. We provide training data but participants are free to use additional data and resources. The only con- straint in the approach is that the processing must be fully automatic — there should be no human intervention. As in past ALTA shared tasks, there are two cat- egories: a student category and an open category. • All the members of teams from the student category must be university students. The teams cannot have members that are full-time employed or that have completed a PhD. • Any other teams fall into the open category. The prize is awarded to the team that performs best on the private test set …

Content truncated to 500 words