Grammatical error detection in transcriptions of spoken English

Proceedings of the 28th International Conference on Computational Linguistics, pages 2144–2162 Barcelona, Spain (Online), December 8-13, 2020 2144 Grammatical error detection in transcriptions of spoken English Andrew Caines1 Christian Bentz2 Kate Knill3 Marek Rei4,1 Paula Buttery1 1 ALTA Institute & Computer Laboratory, University of Cambridge, U.K. {andrew.caines|paula.buttery}@cl.cam.ac.uk 2 Department of General Linguistics, University of Tübingen, Germany chris@christianbentz.de 3 ALTA Institute & Engineering Department, University of Cambridge, U.K. kate.knill@eng.cam.ac.uk 4 Department of Computing, Imperial College London, U.K. marek.rei@imperial.ac.uk Abstract We describe the collection of transcription corrections and grammatical error annotations for the CROWDED Corpus of spoken English monologues on business topics. The corpus recordings were crowdsourced from native speakers of English and learners of English with German as their first language. The new transcriptions and annotations are obtained from different crowdworkers: we analyse the 1108 new crowdworker submissions and propose that they can be used for auto- matic transcription post-editing and grammatical error correction for speech. To further explore the data we train grammatical error detection models with various configurations including pre- trained and contextual word representations as input, additional features and auxiliary objectives, and extra training data from written error-annotated corpora. We find that a model concatenat- ing pre-trained and contextual word representations as input performs best, and that additional information does not lead to further performance gains. 1 Introduction We introduce a new resource for speech-centric natural language processing (speech NLP) – more than a thousand transcriptions and error annotations for 383 distinct recordings from the CROWDED Corpus (Caines et al., 2016). CROWDED is a crowdsourced English corpus of short monologues on business topics, recorded by both native and non-native speakers. It was created in response to the lack of speech corpora freely available for research use, and the lack of appropriate native speaker reference corpora with which language learners’ exam monologues can be compared. In this new project, crowdworkers were asked to first correct existing speech transcriptions and then to edit the resulting transcriptions to make them more fluent. These new annotations enable both post-editing of noisy speech transcriptions – such as might come from automatic speech recognisers – and also grammatical error correction for spoken English. There has been a marked increase in openly-available NLP resources in recent years, especially for English, but these have on the whole been sourced from written texts. Public resources for speech NLP, on the other hand, are relatively scarce, even for English – though English is again by far the best served in this respect. Obtaining linguistic data from large, distributed, online workers (‘crowdsourcing’) has become a well-established practice. With the contribution of these new annotations, we note that the CROWDED Corpus now features a thousand recordings which have all been transcribed, and of which almost 40% now have improved transcriptions and error annotations thanks to this work. The entire corpus has been collated through crowdsourcing means. In this paper we describe the method for collecting the new data, analyse the annotations received, and report on some initial grammatical error detection …

Content truncated to 500 words