Decontextualization: Making Sentences Stand-Alone

Decontextualization: Making Sentences Stand-Alone Eunsol Choi2∗, Jennimaria Palomaki1, Matthew Lamm1, Tom Kwiatkowski1, Dipanjan Das1, Michael Collins1 1Google Research 2Department of Computer Science, The University of Texas at Austin eunsol@cs.utexas.edu, {jpalomaki,mrlamm,tomkwiat,dipanjand,mjcollins}@google.com Abstract Models for question answering, dialogue agents, and summarization often interpret the mean- ing of a sentence in a rich context and use that meaning in a new context. Taking excerpts of text can be problematic, as key pieces may not be explicit in a local window. We isolate and define the problem of sentence decontextual- ization: taking a sentence together with its context and rewriting it to be interpretable out of context, while preserving its meaning. We describe an annotation procedure, collect data on the Wikipedia corpus, and use the data to train models to automatically decontextualize sentences. We present preliminary studies that show the value of sentence decontextualization in a user-facing task, and as preprocessing for systems that performdocument understanding. We argue that decontextualization is an impor- tant subtask in many downstream applications, and that the definitions and resources provided can benefit tasks that operate on sentences that occur in a richer context. 1 Introduction Many applications of natural language processing need to be able to interpret, or present, text inde- pendently from the rich context in which it occurs. For example, summarization systems extract sa- lient information from documents and present it in a reduced context. Many systems also segment documents prior to interpretation of retrieval for computational efficiency. In all of these cases, we would like the context-reduction step to be meaning preserving but, to date, there has been no independent method of ensuring this. ∗Work done at Google. In this paper we isolate and define the problem of sentence decontextualization: taking a sentence together with its context and rewriting it to be interpretable out of context if feasible, while pre- serving its meaning.1 Having defined the problem, we operationalize this definition into a high qual- ity annotation procedure; use the resulting data to train models to automatically decontextualize sentences; and present preliminary results that show the value of automatic decontextualization in a user-facing task, and as preprocessing for systems that perform document understanding. We argue that decontextualization is an important sub-task in many downstream applications, and we believe this work can benefit tasks that operate on sentences that occur in a wider context. One contribution of this work is to release a dataset of decontextualized sentences that can be used as training and evaluation data, together with the evaluation script: On publication of this paper the data will be available at https://github .com/google-research/language/tree /master/language/decontext. Figure 1 shows an example decontextualiza- tion. In this example we have a coreferenceresolu- tion step (their → The Croatia national football team’s) and a bridging step (insertion of the prep- ositional phrase ‘‘in the FIFA World Cup’’ to modify ‘‘Croatia’s best result thus far’’). De- contextualization involves various linguistic phe- nomena, including coreference resolution, global scoping, and bridging anaphora (Clark, 1975). We present a linguistically motivated definition of decontextualiation in …

Content truncated to 500 words