How to Train Dependency Parsers with Inexact Search for Joint Sentence Boundary Detection and Parsing of Entire Documents

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1924–1934, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics How to Train Dependency Parsers with Inexact Search for Joint Sentence Boundary Detection and Parsing of Entire Documents Anders Bj¨orkelund and Agnieszka Fale´nska and Wolfgang Seeker and Jonas Kuhn Institut f¨ur Maschinelle Sprachverarbeitung University of Stuttgart {anders,falensaa,seeker,jonas}@ims.uni-stuttgart.de Abstract We cast sentence boundary detection and syntactic parsing as a joint problem, so an entire text document forms a training instance for transition-based dependency parsing. When trained with an early up- date or max-violation strategy for inexact search, we observe that only a tiny part of these very long training instances is ever exploited. We demonstrate this effect by extending the ArcStandard transition sys- tem with swap for the joint prediction task. When we use an alternative update strat- egy, our models are considerably better on both tasks and train in substantially less time compared to models trained with early update/max-violation. A comparison between a standard pipeline and our joint model furthermore empirically shows the usefulness of syntactic information on the task of sentence boundary detection. 1 Introduction Although punctuation mostly provides reliable cues for segmenting longer texts into sentence units, human readers are able to exploit their un- derstanding of the syntactic and semantic structure to (re-)segment input in the absence of such cues. When working with carefully copy-edited text documents, sentence boundary detection can be viewed as a minor preprocessing task in Natu- ral Language Processing, solvable with very high accuracy. However, when dealing with the out- put of automatic speech recognition or “noisier” texts such as blogs and emails, non-trivial sentence segmentation issues do occur. Dridan and Oepen (2013), for example, show how much impact fully automatic preprocessing can have on parsing qual- ity for well-edited and less-edited text. Two possible strategies to approach this prob- lem are (i) to exploit other cues for sentence boundaries, such as prosodic phrasing and intona- tion in speech (e.g., Kol´aˇr et al. (2006)) or format- ting cues in text documents (Read et al., 2012), and (ii) to emulate the human ability to exploit syntactic competence for segmentation. We focus here on the latter, which has received little atten- tion, and propose to cast sentence boundary detec- tion and syntactic (dependency) parsing as a joint problem, such that segmentations that would give rise to suboptimal syntactic structures can be dis- carded early on. A joint model for parsing and sentence bound- ary detection by definition operates on documents rather than single sentences, as is the standard case for parsing. The task is illustrated in Figure 1, which shows the beginning of a document in the Switchboard corpus, a collection of transcribed telephone dialogues. The parser must predict the syntactic structure of the three sentences as well as the start points of each sentence.1 The simple fact that documents are consider- ably longer than sentences, often by orders of magnitude, creates some interesting challenges for a joint system. First of …

Content truncated to 500 words