A Study on Seq2seq for Sentence Compressionin Vietnamese

A Study on Seq2seq for Sentence Compression in Vietnamese Thi-Trang Nguyen1,2, Huu-Hoang Nguyen2, Kiem-Hieu Nguyen1 1Hanoi Uni. of Science and Technology, Hanoi, Vietnam 2VCCorp, Hanoi, Vietnam {nguyenthitrang03,hoangnguyenhuu}@admicro.vn, hieunk@soict.hust.edu.vn Abstract Text summarization is an important yet chal- lenging task in natural language processing. In this paper, we investigate Pointer Genera- tor Networks for sentence compression. Using Vietnamese as a case study, our model could yield sentence summaries with high quality of syntax, factual correctness and completeness. Interestingly, we demonstrate that only a sim- ple filtering technique is required to generate training data of sentence-summary pairs with- out any human annotation 1 Introduction Text summarization is an important and challeng- ing aspect of natural language processing. The ul- timate goal is to generate summary that retains es- sential information of the original text. Extractive approaches attempts to identify salient parts of the source text and assembles them into a summary. In contrast, abstractive approaches uses language mod- elling technique which is conditioned on original text to generate summary that might have differ- ent sentence structure and novel words. This paper focuses on sentence compression, which could be used in both abstractive and extractive summariza- tion systems or as a stand-alone application. Sen- tence compression methods could be broadly clas- sified into two categories, deletion-based and ab- stractive models. Deletion-based approach depends upon efforts to find and delete unimportant words or phrases in the original sentence. A shorter sen- tence is then produced by stitching together remain- ing fragments. An abstractive sentence compressor, on the other hand, is essentially similar to a docu- ment or multi-document summarizer except that it only takes the original sentence as context and it pro- duces one sentence instead of a multi-sentence sum- mary. Recent success of sequence-to-sequence (seq2seq) framework in machine translation paves the way for the emergence of neural abstrac- tive summarization [Chopra et al.2016, Wubben et al.2016,Nallapati et al.2016,See et al.2017,Rush et al.2015]. Although there have been several studies in Vietnamese text summarization [Nguyen and Nguyen2011, Dac et al.2017], we haven’t been aware of any neural networks-based methods, partially due to the lack of a large volume of high-quality training data. In a newswire article, we observe that sapo1 is typically a longer version of title. Based on this ob- servation, we could collect a large amount of sen- tence - summary pairs from online newswire. It then requires a simple filtering step based on textual sim- ilarity to remove unwanted title - sapo pairs. We rely on a particular model in seq2seq family, i.e. Pointer Generation Networks [See et al.2017] to learn sum- marizing from a large amount of training pairs. As suggested in [Kryscinski et al.2019], we eval- uate generated summaries using human judgement alongside ROUGE metric [Lin2004]. Despite in- evitable noise in training data, our model could yield summaries with high quality of syntax, factual cor- rectness and completeness. 1Sapo is a paragraph, usually containing one sentence, that follows the title and precedes the first sentence of an article. 2 Related work …

Content truncated to 500 words