A Multi- versus a Single-classifier Approach for the Identification of Modality in the Portuguese Language

A Multi- versus a Single-classifier Approach for the Identification of Modality in the Portuguese Language Jo˜ao Sequeira1, Teresa Gonc¸alves1, Paulo Quaresma1, Am´alia Mendes2, Iris Hendrickx23 1Department of Informatics, University of ´Evora, Portugal 2Center of Linguistics, University of Lisbon, Portugal 3Centre for Language Studies, Radboud University Nijmegen, The Netherlands d11594@alunos.uevora.pt, tcg@uevora.pt, pq@uevora.pt, amaliamendes@letras.ulisboa.pt, iris@i-hx.nl Abstract This work presents a comparative study between two different approaches to build an automatic classification system for Modality values in the Portuguese language. One approach uses a single multi-class classifier with the full dataset that includes eleven modal verbs; the other builds different classifiers, one for each verb. The performance is measured using precision, recall and F1. Due to the unbalanced nature of the dataset a weighted average approach was calculated for each metric. We use support vector machines as our classifier and experimented with various SVM kernels to find the optimal classifier for the task at hand. We experimented with several different types of feature attributes representing parse tree information and compare these complex feature representation against a simple bag-of-words feature representation as baseline. The best obtained F1 values are above 0.60 and from the results it is possible to conclude that there is no significant difference between both approaches. Keywords: Natural Language Processing, Modality, Feature Selection, Support Vector Machines 1. Introduction In the last years there was a great development in fields related to Machine Learning in the pursuit of forms of Arti- ficial Intelligence (AI), which has become a major research trend of both academic and companies, like Google and Mi- crosoft. The fields are diverse, ranging from financial fraud identification, image recognition and even systems that can rewrite their own code or write other programs (Caughill, 2017; Gershgorn, 2017; Galeon, 2017). Natural Language Processing (NLP) is a related research field that includes tasks aiming at understanding texts, information extraction and text classification. Presently, one of the most active sub-field focuses on sen- timent analysis and opinion mining (Pang and Lee, 2008). This includes tasks such as the (automatic) distinction be- tween the factual and non-factual nature of events and the detection of the subjective perspective underlying texts. Modality is one indicator of subjectivity and factuality and it is usually defined as the expression of the speaker’s opin- ion and attitude towards the proposition (Palmer, 1986). Traditionally, it covers epistemic modality, which is related to the degree of commitment of the speaker to the truth of the proposition (whether the event is perceived as possi- ble, probable or certain), but also deontic modality (obliga- tion or permission), capacity and volition (Sequeira et al., 2016). Information about the modality of a text is crucial for the above mentioned trends on automatic fact finding and information extraction. This work extends the experiments done previously (Se- queira et al., 2016) in the pursuit of creating a semi- automatic modality tagging system for the Portuguese lan- guage from a manually annotated corpus that uses the modality scheme described by Hendrickx et al. (2012) and Mendes et al. …

Content truncated to 500 words