Exploiting Parse Structures for Native Language Identification

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 1600–1610, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics Exploiting Parse Structures for Native Language Identification Sze-Meng Jojo Wong Centre for Language Technology Macquarie University Sydney, Australia sze.wong@mq.edu.au Mark Dras Centre for Language Technology Macquarie University Sydney, Australia mark.dras@mq.edu.au Abstract Attempts to profile authors according to their characteristics extracted from textual data, in- cluding native language, have drawn attention in recent years, via various machine learn- ing approaches utilising mostly lexical fea- tures. Drawing on the idea of contrastive analysis, which postulates that syntactic er- rors in a text are to some extent influenced by the native language of an author, this paper explores the usefulness of syntactic features for native language identification. We take two types of parse substructure as features— horizontal slices of trees, and the more gen- eral feature schemas from discriminative parse reranking—and show that using this kind of syntactic feature results in an accuracy score in classification of seven native languages of around 80%, an error reduction of more than 30%. 1 Introduction Inferring characteristics of authors from their tex- tual data, often termed authorship profiling, has seen a number of computational approaches proposed in recent years. The problem is typically treated as a classification task, where an author is classified with respect to characteristics such as gender, age, native language, and so on. This profile information is of- ten of interest to marketing organisations for prod- uct promotional reasons as well as governments or law enforcements for crime investigation purposes. The particular application that motivates the present study is detection of phishing (Myers, 2007), the at- tempt to defraud through texts that are designed to deceive Internet users into giving away confidential details. One class of countermeasures to phishing consists of technical methods such as email authen- tication; another looks at profiling of the text’s au- thor(s) (Fette et al., 2007; Zheng et al., 2003), to find any indications of the source of the text. In this paper we investigate classification of a text with respect to an author’s native language, where this is not the language that that text is written in (which is often the case in phishing); we refer to this as native language identification. Initial work by Koppel et al. (2005) was followed by Tsur and Rappoport (2007), Estival et al. (2007), van Halteren (2008), and Wong and Dras (2009). By and large, the problem was tackled using various supervised machine learning approaches, with mostly lexical features over characters, words, and parts of speech, as well as some document structure. Syntactic features, in contrast, in particular those that capture grammatical errors, which might po- tentially be useful for this task, have received lit- tle attention. Koppel et al. (2005) did suggest using syntactic errors in their work but did not investigate them in any detail. Wong and Dras (2009) noted the relevance of the concept of contrastive analy- sis (Lado, 1957), which postulates that …

Content truncated to 500 words