Automating Second Language Acquisition Research: Integrating Information Visualisation and Machine Learning

Proceedings of the EACL 2012 Joint Workshop of LINGVIS & UNCLH, pages 35–43, Avignon, France, April 23 - 24 2012. c⃝2012 Association for Computational Linguistics Automating Second Language Acquisition Research: Integrating Information Visualisation and Machine Learning Helen Yannakoudakis Computer Laboratory University of Cambridge United Kingdom Helen.Yannakoudakis@cl.cam.ac.uk Ted Briscoe Computer Laboratory University of Cambridge United Kingdom Ted.Briscoe@cl.cam.ac.uk Theodora Alexopoulou DTAL University of Cambridge United Kingdom ta259@cam.ac.uk Abstract We demonstrate how data-driven ap- proaches to learner corpora can support Second Language Acquisition research when integrated with visualisation tools. We present a visual user interface support- ing the investigation of a set of linguistic features discriminating between pass and fail ‘English as a Second or Other Lan- guage’ exam scripts. The system displays directed graphs to model interactions between features and supports exploratory search over a set of learner scripts. We illustrate how the interface can support the investigation of the co-occurrence of many individual features, and discuss how such investigations can shed light on understanding the linguistic abilities that characterise different levels of attainment and, more generally, developmental aspects of learner grammars. 1 Introduction The Common European Framework of Reference for Languages (CEFR)1 is an international bench- mark of language attainment at different stages of learning. The English Profile (EP)2 research pro- gramme aims to enhance the learning, teaching and assessment of English as an additional lan- guage by creating detailed reference level descrip- tions of the language abilities expected at each level. As part of our research within that frame- work, we modify and combine techniques devel- oped for information visualisation with method- ologies from computational linguistics to support a novel and more empirical perspective on CEFR 1http://www.coe.int/t/dg4/linguistic/cadre en.asp 2http://www.englishprofile.org/ levels. In particular, we build a visual user in- terface (hereafter UI) which aids the develop- ment of hypotheses about learner grammars us- ing graphs of linguistic features discriminating pass/fail exam scripts for intermediate English. Briscoe et al. (2010) use supervised discrimi- native machine learning methods to automate the assessment of ‘English as a Second or Other Lan- guage’ (ESOL) exam scripts, and in particular, the First Certificate in English (FCE) exam, which assesses English at an upper-intermediate level (CEFR level B2). They use a binary discrimina- tive classifier to learn a linear threshold function that best discriminates passing from failing FCE scripts, and predict whether a script can be clas- sified as such. To facilitate learning of the clas- sification function, the data should be represented appropriately with the most relevant set of (lin- guistic) features. They found a discriminative fea- ture set includes, among other feature types, lexi- cal and part-of-speech (POS) ngrams. We extract the discriminative instances of these two feature types and focus on their linguistic analysis3. Ta- ble 1 presents a small subset ordered by discrimi- native weight. The investigation of discriminative features can offer insights into assessment and into the linguis- tic properties characterising the relevant CEFR level. However, the amount and variety of data potentially made available by the classifier is con- siderable, as …

Content truncated to 500 words