An Empirical Analysis of Edit Importance between Document Versions

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2780–2784 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics An Empirical Analysis of Edit Importance between Document Versions Tanya Goyal tanyagoyal.93@gmail.com Big Data Experience Lab Adobe Research Sachin Kelkar sachinkel19@gmail.com Indian Institute of Technology, Roorkee Manas Agarwal manasagarwal1993@gmail.com Indian Institute of Technology, Roorkee Jeenu Grover groverjeenu@gmail.com Indian Institute of Technology, Kharagpur Abstract In this paper, we present a novel approach to infer significance of various textual ed- its to documents. An author may make several edits to a document; each edit varies in its impact to the content of the document. While some edits are surface changes and introduce negligible change, other edits may change the content/tone of the document significantly. In this pa- per, we perform an analysis of the human perceptions of edit importance while re- viewing documents from one version to the next. We identify linguistic features that influence edit importance and model it in a regression based setting. We show that the predicted importance by our approach is highly correlated with the human per- ceived importance, established by a Me- chanical Turk study. 1 Introduction In collaborative content authoring, multiple au- thors make changes to the same document, which results in the final version being significantly dif- ferent from the base draft. Often there is a need to review the edits made to the original document, which can be a long and arduous task. Tools like Microsoft Word (mic) and Adobe Acrobat (ado) provide reviewers with a list of edits, in the form of insertions and deletions. While helpful, these tools do not differentiate between the different types of edits, or consider the varying impact of edits. For instance, change from numeric ‘18’ to word ‘eigh- teen’ may be a minor change and less crucial for the author to review, as compared to an edit that All authors have equal contribution in this paper. This work was done as part of an internship at Adobe Research alters the facts of the document. Thus, in our work, we focus on automatically inferring the im- pact/change introduced by edits, and predict the perceived importance of such edits by authors. In this paper, we perform a linguistic analysis of how humans evaluate the significance of edits while reviewing documents. Our algorithm as- signs scores to edits between two versions of a document, which indicate the significance of the specified edit as perceived by the reviewer. We demonstrate the efficacy of our approach by com- paring our algorithm generated edit importance scores with the human perceived ground truth im- portance, established through a Mechanical Turk survey. 2 Related Work There has been significant amount of work on defining the importance of a keyword or a sentence in the context of document summarization (Mi- halcea and Tarau, 2004). Some prior work has also been done on inferring the type of edits between Wikipedia versions. Bronner et al. (Bronner and Monz, 2012) proposed a supervised approach to …

Content truncated to 500 words