Topic Modeling for Native Language Identification

Topic Modeling for Native Language Identification Sze-Meng Jojo Wong Mark Dras Mark Johnson Centre for Language Technology Macquarie University Sydney, NSW, Australia {sze.wong,mark.dras,mark.johnson}@mq.edu.au Abstract Native language identification (NLI) is the task of determining the native language of an author writing in a second language. Several pieces of earlier work have found that fea- tures such as function words, part-of-speech n-grams and syntactic structure are helpful in NLI, perhaps representing characteristic er- rors of different native language speakers. This paper looks at the idea of using Latent Dirichlet Allocation as a feature clustering technique over lexical features to see whether there is any evidence that these smaller-scale features do cluster into more coherent latent factors, and investigates their effect in a clas- sification task. We find that although (not un- expectedly) classification accuracy decreases, there is some evidence of coherent clustering, which could help with much larger syntactic feature spaces. 1 Introduction Native language identification (NLI), the task of de- termining the native language of an author writing in a second language, typically English, has gained in- creased attention in recent years. The problem was first phrased as a text classification task by Koppel et al. (2005), using a machine learner with fundamen- tally lexical features — function words, character n- grams, and part-of-speech (PoS) n-grams. A number of subsequent pieces of work, such as that of Tsur and Rappoport (2007), Estival et al. (2007), Wong and Dras (2009) and Wong and Dras (2011), have taken that as a starting point, typically along with a wider range of features, such as document structure or syntactic structure. Wong and Dras (2011) looked particularly at syn- tactic structure, in the form of production rules and parse reranking templates. They noted that they did not find the expected instances of clearly un- grammatical elements of syntactic structure indicat- ing non-native speaker errors; instead there were just different distributions over regular elements of grammatical structure for different native languages. Our intuition is that it is several elements together that indicate particular kinds of indicative errors, such as incorrect noun-number agreement; and from this, that there might be coherent clusters of corre- lated features that are indicative of a particular native language. In this preliminary work, we investigate this using the basic lexical features of the original Koppel et al. (2005) model. Latent Dirichlet Allocation (LDA) — a genera- tive probabilistic model for unsupervised learning — was first introduced by Blei et al. (2003) to dis- cover a set of latent mixture components known as topics which are representative of a collection of discrete data. The underlying idea of LDA is that each document from a text corpus is constructed ac- cording to a specific distribution of topics, in which words comprising the document are generated based on the word distribution for each selected topic; a topic is typically represented by a set of words such as species, phylogenetic, evolution and so on. Such a model allows multiple topics in one document as well as sharing …

Content truncated to 500 words