Identifying Authors Based on Stylometric measures of Vietnamese texts

Exploiting weak-supervision for classifying Non-Sentential Utterances in Mandarin Conversations Xin-Yi Chen The Hong Kong Polytechnic University Department of Chinese and Bilingual Studies xysimba.chen@connect.polyu.hk Laurent Pr´evot Aix Marseille Universit´e & CNRS Laboratoire Parole et Langage laurent.prevot@univ-amu.fr Abstract Non-sentential or fragmentary utterances (NSU) constitute a significant part of the productions in a conversation. Although seemingly incomplete in form, they convey full pragmatic meaning in the context. In the past, their classification had been approached with supervised methods (Fern´andez et al., 2007; Wong, 2018). Such approaches require relatively large annotated data sets. We explore an approach (Ratner et al., 2017a) that allows the reduce significantly the amount of annotated data needed thanks to strategic use of linguistic knowledge. We explore this method for classifying NSUs in Mandarin conversation corpus. Our evaluation shows that promising results can be obtained with a minimal amount of annotated training data. 1 Introduction In dialogue, besides well-formed complete sen- tences, a sizeable amount of utterances are frag- ments that could be understood without a problem in their context. Traditional grammar attends mainly to written texts and canonical sentence analysis. The oral language has been often regarded as bad, spon- taneous, and wrong, in summary not an appropriate research object, as (Blanche-Benveniste, 1997) re- grets it. But as interest in oral communication gets more attention, terms like “fragments”, “Nonsenten- tials” in (Barton, 1991) or “Non Sentential Utter- ances” (hereafter NSU) in (Fern´andez et al., 2007) have also attracted more investigation. The expressions in example 1 below may sound familiar. (1) What now? Not you. What’s for supper? - Ground Beef Tacos. Even though they are generally short, such utter- ances constitute an active part of the conversation. They contribute to the efficiency of the conversa- tion flow. The interpretation of NSU is essential for linguistic theories that attempt to get serious about language as it is produced in its most natural and pervasive setting, and also for applications, like di- alogue systems. It can be done in different ways, as discussed in (Ginzburg, 2012, p:229). The anal- ysis result can be implemented in human-machine dialogue systems in various domains such as client service or computer aided language teacher. The percentage of NSU among other utterances in conversation corpus is non negligible, 11.15 % in (Fern´andez and Ginzburg, 2002), 9% in (Fern´andez et al., 2007), 10.2 % in (Schlangen and Lascarides, 2003). We think the study of NSUs is useful because of the high frequency mentioned above. What’s more, the understanding of NSUs and their classifi- cation from their context is not always easy. Even a simple “what” can express various emotions and can have different functions in a context. Apart from the most common function as plain question, it can also express Happiness, Surprise, Sadness, Anger, Dis- gust or even Fear. Second, the definition of NSUs can have an im- pact on the classification of NSU, the inclusion and exclusion of categories can be flexible according to the theories and purpose of classification, the classi- fication criteria could be …

Content truncated to 500 words