Classifying Judgements using Transfer Learning

Classifying JUDGEMENTS using Transfer Learning Pradeesh Parameswaran, Andrew Trotman, Veronica Liesaputra, David Eyers Department of Computer Science University of Otago New Zealand [pradeesh,andrew,veronica,dme]@cs.otago.ac.nz Abstract We describe our method for classifying short texts into the APPRAISAL framework, work we conducted as part of the ALTA 2020 shared task. We tackled this problem using trans- fer learning. Our team, “orangutanV2” placed equal first in the shared task, with a mean F1- score of 0.1026 on the private data set. 1 Introduction Systemic Functional Linguistics (SFL) is a the- ory of language which examines the relationship between language meaning and the functions in a social context (Halliday, 1996). One popular framework that uses SFL is APPRAISAL (Mar- tin and White, 2005). The APPRAISAL frame- work is based on the notion of uncovering the at- titude of the author from the perspective of a po- tential listener or reader. It is used by linguists in analysing human behaviour from textual data (Ross and Caldwell, 2020; Starfield et al., 2015; Wu, 2013; Hommerberg and Don, 2015). Figure 1 shows an overview of the APPRAISAL framework. The three main resources of the APPRAISAL framework are; ATTITUDE, ENGAGEMENT and GRADUATION (Martin and White, 2005). The AT- TITUDE framework is then subdivided into three subsystems; AFFECT (emotions), APPRECIATION (evaluation of natural and semiotic phenomena) and JUDGEMENT (evaluation of people and their behaviour). The JUDGEMENT subsystem can be divided into two categories: SOCIAL ESTEEM and SOCIAL SANCTIONS. SOCIAL ESTEEM primar- ily involves admiration and criticism and SOCIAL SANCTION involves praise and condemnation. SOCIAL SANCTIONS can be further divided into three subcategories: normality (how usual one is), capacity (how capable one person is) and tenacity (how dependable one is). As for SOCIAL SANC- TION it can be further divided into two subcate- APPRAISAL Framework ENGAGEMENT ATTITUDE APPRECIATION JUDGEMENT AFFECT GRADUATION SOCIAL ESTEEM SOCIAL SANCTION Normality Capacity Tenacity Veracity Propriety Figure 1: The APPRAISAL Framework (Adapted From (Martin and White, 2005)) gories; veracity (how truthful one is) and propriety (how virtuous one is). The robustness of the APPRAISAL framework lies in its ability to be used in various different social contexts. It also offers linguists detailed strategies for realising the framework (Ngo and Unsworth, 2015). Since its debut, the APPRAISAL framework has been widely used to explore how language is being used in various different envi- ronments such as in analysing examiners’ reports on doctoral theses (Starfield et al., 2015), Don- ald Trump’s rhetoric tweets (Ross and Caldwell, 2020), people’s perception on the outcome of the Brexit referendum (Bouko and Garcia, 2020) and in teaching English as a second language (ESL) (Ngo and Unsworth, 2015). Currently, linguists manually classify sentences using annotation software as there is no automated classification technique that exists to automate the task (Fuoli, 2018). Thus, this problem sparked the interest of Australasian Technology Associa- tion (ALTA) to organise a shared challenge task to develop a model that can automatically identify and classify human behaviour (JUDGEMENT) ex- Feature matched Number (None) 104 Normality 22 Capacity 31 Tenacity 21 …

Content truncated to 500 words