Incremental Natural Language Processing: Challenges, Strategies, and Evaluation

Proceedings of the 27th International Conference on Computational Linguistics, pages 2990–3003 Santa Fe, New Mexico, USA, August 20-26, 2018. 2990 Incremental Natural Language Processing: Challenges, Strategies, and Evaluation Arne K¨ohn Natural Lanuguage Systems Group Department of Informatics Universit¨at Hamburg koehn@informatik.uni-hamburg.de Abstract Incrementality is ubiquitous in human-human interaction and beneficial for human-computer interaction. It has been a topic of research in different parts of the NLP community, mostly with focus on the specific topic at hand even though incremental systems have to deal with similar challenges regardless of domain. In this survey, I consolidate and categorize the approaches, identifying similarities and differences in the computation and data, and show trade-offs that have to be considered. A focus lies on evaluating incremental systems because the standard metrics often fail to capture the incremental properties of a system and coming up with a suitable evaluation scheme is non-trivial. Title and Abstract in German Inkrementelle Sprachverarbeitung: Herausforderungen, Strategien und Evaluation Inkrementalit¨at ist allgegenw¨artig in Mensch-Mensch-Interaktiton und hilfreich f¨ur Mensch- Computer-Interaktion. In verschiedenen Teilen der NLP-Community wird an Inkrementalit¨at geforscht, zumeist fokussiert auf eine konkrete Aufgabe, obwohl sich inkrementellen Systemen dom¨anen¨ubergreifend ¨ahnliche Herausforderungen stellen. In diesem ¨Uberblick trage ich Ans¨atze zusammen, kategorisiere sie und stelle ¨Ahnlichkeiten und Unterschiede in Berechnung und Daten sowie n¨otige Abw¨agungen vor. Ein Fokus liegt auf der Evaluierung inkrementeller Systeme, da Standardmetriken of nicht in der Lage sind, die inkrementellen Eigenschaften eines Systems einzufangen und passende Evaluationsschemata zu entwickeln nicht einfach ist. 1 Introduction Interaction using language is incremental in many forms: In a dialogue, understanding takes place continuously and participants are able to interrupt each other or signal whether they understand the currently ongoing utterance. Simultaneous interpreters translate speeches as they are spoken. Texts are (partially) understood before they are fully read. Incremental processing exploits this property by starting to compute before all input is available, allowing a system to already act on partial input. Without incremental processing, an NLP system is unable to perform any of these tasks as it is bound to wait for complete utterances or a finished text and only afterwards it can compute how to (re-)act, leading to deficiencies in human-computer interaction. First, I want to delimit this notion of incrementality from other notions: Systems that work on a complete input but generate output layer by layer, e.g. shallow to deep syntax, are sometimes called incremental, e.g. by A¨ıt-Mokhtar et al. (2002). These systems are not discussed in this survey. Anytime algorithms are incremental in the sense that they iteratively improve their output for a fixed input, given more and more processing time. They can play an important role in incremental systems as they allow to This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/. 2991 perform a trade-off between processing time – i.e. system responsiveness – and quality. The question of whether to employ anytime algorithms is however orthogonal to the approaches discussed in this survey1. In an incremental system, all processors need to work incrementally. A …

Content truncated to 500 words