Advancing Linguistic Features and Insights by Label-informed Feature Grouping: An Exploration in the Context of Native Language Identification

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 739–749, Osaka, Japan, December 11-17 2016. Advancing Linguistic Features and Insights by Label-informed Feature Grouping: An Exploration in the Context of Native Language Identification Serhiy Bykh Seminar f¨ur Sprachwissenschaft Universit¨at T¨ubingen sbykh@sfs.uni-tuebingen.de Detmar Meurers Seminar f¨ur Sprachwissenschaft Universit¨at T¨ubingen dm@sfs.uni-tuebingen.de Abstract We propose a hierarchical clustering approach designed to group linguistic features for super- vised machine learning that is inspired by variationist linguistics. The method makes it possible to abstract away from the individual feature occurrences by grouping features together that be- have alike with respect to the target class, thus providing a new, more general perspective on the data. On the one hand, it reduces data sparsity, leading to quantitative performance gains. On the other, it supports the formation and evaluation of hypotheses about individual choices of linguis- tic structures. We explore the method using features based on verb subcategorization information and evaluate the approach in the context of the Native Language Identification (NLI) task. 1 Introduction and related work Native Language Identification (NLI) is the task of inferring the native language (L1) of writers from texts they wrote in another language. NLI started to attract attention in computational linguistics with the work of Koppel et al. (2005). Since then interest has steadily risen, leading to the First NLI Shared Task in 2013, with 29 participating teams (Tetreault et al., 2013). NLI is usually considered as a text classification problem with the different L1s as labels. A range of features reaching from character and word n-grams to dependency- and constituency-based features have successfully been used in standard supervised machine learning setups, yielding accuracies of up to around 83% for the 11 classes in the First NLI Shared Task. Some more recent papers further advance the best result from that competition, namely 83.6% (Jarvis et al., 2013), reaching around 85% (Bykh and Meurers, 2014; Ionescu et al., 2014). While pushing the quantitative side is one option of advancing the NLI work further, another avenue of research tries to improve our understanding of how the different feature types work and what conclu- sions one can draw from these observations for Second Language Acquisition (SLA) research. Swanson and Charniak (2013) utilized Tree Substitution Grammars as well as different measures of relevancy and redundancy to extract indicative linguistic patterns. Swanson and Charniak (2014) adopted the approach to dependencies. Malmasi and Dras (2014) proposed a technique to detect over- and underuse of certain patterns by writers with a particular L1-background using linear SVM weights derived from Adaptor grammar collocations or Stanford Dependencies. Meurers et al. (2014) employed verb subcategorization patterns as features and showed that there are differences in the usage patterns of verbs between native English writers and, e.g., writers with Chinese L1-background. Bykh and Meurers (2014) systemati- cally explored constituency-based features and discussed the distinctive power of the different variants realizing lexical and phrasal categories. Malmasi and Cahill (2015) investigated the correlation between various features in a feature …

Content truncated to 500 words