Classifying Sluice Occurrences in Dialogue

Classifying Sluice Occurrences in Dialogue Austin Baird, Anissa Hamza, Daniel Hardt UCSC, LiLPa/Unistra, Copenhagen Business School/UCSC arbaird@ucsc.edu, anissa.hamza@etu.unistra.fr, dhardt@ucsc.edu Abstract Ellipsis is an important challenge for natural language processing systems, and addressing that challenge requires large collections of relevant data. The dataset described by Anand and McCloskey (2015), consisting of 4100 occurrences, is an important step towards addressing this issue. However, many NLP technologies require much larger collections of data. Furthermore, previous collections of ellipsis are primarily restricted to news data, although sluicing presents a particularly important challenge for dialogue systems. In this paper we classify sluices as Direct, Reprise, Clarification. We perform manual annotation with acceptable inter-coder agreement. We build classifier models with Decision Trees and Naive Bayes, with accuracy of 67%. We deploy a classifier to automatically classify sluice occurrences in OpenSubtitles, resulting in a corpus with 1.7 million occurrences. This will support empirical research into sluicing in dialogue, and it will also make it possible to build NLP systems using very large datasets. This is a noisy dataset; based on a small manually annotated sample, we found that only 80% of instances are in fact sluices, and the accuracy of sluice classification is lower. Despite this, the corpus can be of great use in research on sluicing and development of systems, and we are making the corpus freely available on request. Furthermore, we are in the process of improving the accuracy of sluice identification and annotation for the purpose of created a subsequent version of this corpus. Keywords: sluicing, ellipsis, dialogue 1. Introduction Ellipsis is a major challenge for NLP systems, as well as an important topic in theoretical linguistics. The most exten- sive empirical work to date on ellipsis is described in Anand and Hardt (2016) and Anand and McCloskey (2015). This work involves a corpus of some 4100 sluice occurrences, extracted from the NYTimes Gigaword Corpus. These oc- currences have been manually annotated in a detailed fash- ion. Sluices are elliptical questions, where all but the interrog- ative phrase of a question is omitted, leaving a wh-word remnant, as in the following example, with the sluice wh- word in bold (Anand and Hardt, 2016): (1) Harry traveled to southern Denmark to study botany. I want to know why. In this paper, we construct a very large corpus of sluice occurrences in dialog. We build on previous work (Anand and McCloskey, 2015; Fernández et al., 2004; Fernández et al., 2007) in developing methods to automatically identify and classify sluice occurrences. We apply these methods to the English portion of OpenSubtitles, resulting in a corpus of over 1.7 million sluice occurrences. This is orders of magnitude larger that any previous collections of ellipsis occurrences and it has been automatically annotated with linguistically relevant features. 2. Related Work (Fernández et al., 2004; Fernández et al., 2007) describe an approach to the classification of sluice occurrences in the British National Corpus (BNC). Fernandez et al. focus on what they call bare sluices: utterances in dialog consist- ing of only …

Content truncated to 500 words