TableGPT: Few-shot Table-to-Text Generation with Table Structure Reconstruction and Content Matching
4176Authors
nan
Abstract
AbstractAlthough neural table-to-text models have achieved remarkable progress with the help of large-scale datasets, they suffer insufficient learning problem with limited training data. Recently, pre-trained language models show potential in few-shot learning with linguistic knowledge learnt from pretraining on large-scale corpus. However, benefiting table-to-text generation in few-shot setting with the powerful pretrained language model faces three challenges, including (1) the gap between the task’s structured input and the natural language input for pretraining language model. (2) The lack of modeling for table structure and (3) improving text fidelity with less incorrect expressions that are contradicting to the table. To address aforementioned problems, we propose TableGPT for table-to-text generation. At first, we utilize table transformation module with template to rewrite structured table in natural language as input for GPT-2. In addition, we exploit multi-task learning with two auxiliary tasks that preserve table’s structural information by reconstructing the structure from GPT-2’s representation and improving the text’s fidelity with content matching task aligning the table and information in the generated text. By experimenting on Humans, Songs and Books, three few-shot table-to-text datasets in different domains, our model outperforms existing systems on most few-shot settings.
Keywords
Full Content
Proceedings of the 28th International Conference on Computational Linguistics, pages 198–207 Barcelona, Spain (Online), December 8-13, 2020 198 End-to-End Emotion-Cause Pair Extraction with Graph Convolutional Network Ying Chen1 Wenjun Hou1 Shoushan Li2 Caicong Wu1 Xiaoqiang Zhang1 1College of Information and Electrical Engineering, China Agricultural University, China 2Natural Language Processing Lab, Soochow University, China {chenying,houwenjun,wucc,xqzhang}@cau.edu.cn lishoushan@suda.edu.cn Abstract Emotion-cause pair extraction (ECPE), which aims at simultaneously extracting emotion-cause pairs that express emotions and their corresponding causes in a document, plays a vital role in understanding natural languages. Considering that a cause usually appears around its correspond- ing emotion, we construct a pair graph and a Pair Graph Convolutional Network (PairGCN) to model dependency relations among local neighborhood candidate pairs. Moreover, in our pro- posed graph, there are three types of dependency relations and each type of dependency relations has its own way to propagate contextual information. Experiments on a benchmark Chinese emotion-cause pair extraction corpus demonstrate the effectiveness of the proposed model. 1 Introduction Emotion-cause pair extraction (ECPE), which was first proposed in Xia & Ding (2019), aims to ex- tract emotion expressions and their corresponding causes in a document simultaneously. Different from emotion cause extraction (ECE) (Lee et al., 2010; Gui et al., 2016) which extracts the causes for given emotion expressions, EPCE is a much more challenging task. There has been a surging interest in developing neural models either for emotion cause extraction or for emotion extraction, while ECPE, as a special causal relation extraction task, is newly proposed and remains largely unexplored. Previous research on ECPE (Xia and Ding, 2019) focused on designing pipeline systems in which emotion clauses and cause clauses are extracted separately, and the two sets of clauses are paired to generate candidate emotion-cause pairs, and then emotion-cause pairs are selected from these candidate pairs with a filter. Hence, prediction errors unavoidably accumulate through the pipeline framework. Therefore, in this work, we aim to design an end-to-end framework in which any two clauses in a document are paired (i.e., one is a candidate emotion clause, and the other is a candidate cause clause) so as to generate candidate emotion-cause pairs and then emotion-cause pairs are selected from these candidate pairs. E.g., in Fig. 1, there are 25 candidate pairs, and only (c4, c2) and (c4, c3) are emotion-cause pairs. Furthermore, modelling contextual information for a candidate pair is also crucial for ECPE. In previ- ous pipeline systems (Xia and Ding, 2019), two features were extracted for each clause in a document for emotion extraction and emotion cause extraction respectively, and then a candidate pair was represented by the combination of these clause-level features. However, dependency among candidate pairs does not take into account. In fact, an emotion usually has a few cause clauses occurring within a specific distance from the emotion expression. E.g, in the Chinese ECPE corpus provided by Xia & Ding (2019), ∼90% emotions has one and only one cause clause, and ∼96% cause clauses occur within a window size of 2 from their …
Content truncated to 500 words
Labeled Text
AbstractAlthough neural table-to-text models have achieved remarkable progress with the help of large-scale datasets, they suffer insufficient learning problem with limited training data. Recently, pre-trained language models show potential in few-shot learning with linguistic knowledge learnt from pretraining on large-scale corpus. However, benefiting table-to-text generation in few-shot setting with the powerful pretrained language model faces three challenges, including (1) the gap between the task’s structured input and the natural language input for pretraining language model. (2) The lack of modeling for table structure and (3) improving text fidelity with less incorrect expressions that are contradicting to the table. To address aforementioned problems, we propose TableGPT for table-to-text generation. At first, we utilize table transformation module with template to rewrite structured table in natural language as input for GPT-2. In addition, we exploit multi-task learning with two auxiliary tasks that preserve table’s structural information by reconstructing the structure from GPT-2’s representation and improving the text’s fidelity with content matching task aligning the table and information in the generated text. By experimenting on Humans, Songs and Books, three few-shot table-to-text datasets in different domains, our model outperforms existing systems on most few-shot settings.
Publication Details
- Paper ID:
- 4176
- Year:
- 2020
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan