Linguistic profiling of texts for the purpose of language verification

Linguistic profiling of texts for the purpose of language verification Hans VAN HALTEREN Dept. of Language and Speech, Univ.of Nijmegen, The Netherlands P.O. Box 9103, 6500 HD Nijmegen hvh@let.kun.nl Nelleke OOSTDIJK Dept. of Language and Speech, Univ. of Nijmegen, The Netherlands P.O. Box 9103, 6500 HD Nijmegen n.oostdijk@let.kun.nl Abstract In order to control the quality of internet-based language corpora, we developed a method to verify automatically that texts are of (near-) native quality. For the LOCNESS and ICLE corpora, the method is rather successful in separating native and non-native learner texts. The Equal Error Rate is about 10%. However, for other domains, such as internet texts, separate classifiers have to be trained on the basis of suitable seed corpora. 1 Introduction Research in linguistics and language engineering thrives on the availability of data. Traditionally, corpora would be compiled with a specific purpose in mind. Such corpora characteristically were well- balanced collections of data. In the form of metadata, record was kept of the design criteria, sampling procedures, etc. Thus the researcher would have a fair idea of where his data originated from. Over past decades, data collection has been boosted by technological developments. More and more and increasingly large collections of data have been and are being compiled. It is tempting to think that the problem of data sparseness has been solved – at least for raw data or data without any annotation other than can be provided fully automatically – especially now that large amounts of data can be accessed through the internet. However, with data coming to us from all over the world, originating from all sorts of sources, we now possibly have a new problem on our hands: often the origins of the found data remain obscure. It is not always clear what exactly the implications for our research are of employing data whose origin we do not know. Is it legal to use these data, ethical, appropriate, …? In this paper we will focus on the last point: the appropriateness of the data in the light of a specific application or research goal. More in particular, we will investigate to what extent we can devise a procedure that will enable us to identify texts produced by native speakers of the language (and thus by default those produced by non-native speakers). The present study is motivated by the fact that for many uses the (near-)nativeness of the data is a critical factor in the development of adequate resources and applications. Thus, for example, a style checker or some other writing assistant tool which has been based on erroneous materials or at least materials deviant from the language targeted, will not always respond appropriately. 1.1 Assessing (near-)nativeness In the general absence of metadata which attest that texts have been produced by native speakers, there is one obvious approach that one may consider in order to assess the (near-)nativeness of texts of unknown origin and that is to exploit their specific linguistic characteristics. Previous studies investigating language variation (eg …

Content truncated to 500 words