MReD: A Meta-Review Dataset for Structure-Controllable Text Generation

Findings of the Association for Computational Linguistics: ACL 2022, pages 2521 - 2535 May 22-27, 2022 c⃝2022 Association for Computational Linguistics MReD: A Meta-Review Dataset for Structure-Controllable Text Generation Chenhui Shen∗ †1,2 Liying Cheng∗† 1,3 Ran Zhou†1,4 Lidong Bing‡1 Yang You2 Luo Si1 1DAMO Academy, Alibaba Group 2 National University of Singapore 3Singapore University of Technology and Design 4 Nanyang Technological University {chenhui.shen, liying.cheng, ran.zhou}@alibaba-inc.com {l.bing, luo.si}@alibaba-inc.com youy@comp.nus.edu.sg Abstract When directly using existing text generation datasets for controllable generation, we are facing the problem of not having the domain knowledge and thus the aspects that could be controlled are limited. A typical example is when using CNN/Daily Mail dataset for con- trollable text summarization, there is no guided information on the emphasis of summary sen- tences. A more useful text generator should leverage both the input text and the control sig- nal to guide the generation, which can only be built with a deep understanding of the domain knowledge. Motivated by this vision, our pa- per introduces a new text generation dataset, named MReD. Our new dataset consists of 7,089 meta-reviews and all its 45k meta-review sentences are manually annotated with one of the 9 carefully defined categories, including ab- stract, strength, decision, etc. We present exper- imental results on start-of-the-art summariza- tion models, and propose methods for structure- controlled generation with both extractive and abstractive models using our annotated data. By exploring various settings and analyzing the model behavior with respect to the control signal, we demonstrate the challenges of our proposed task and the values of our dataset MReD. Meanwhile, MReD also allows us to have a better understanding of the meta-review domain. 1 1 Introduction Text generation entered a new era because of the development of neural network based generation techniques. Along the dimension of the mapping relation between the input information and the out- put text, we can roughly group the recent tasks ∗ Equally Contributed. †Chenhui, Liying, and Ran are under the Joint PhD Pro- gram between Alibaba and their corresponding universities. ‡ Corresponding author. 1Our code and data are released at https://github. com/Shen-Chenhui/MReD. meta-review: [This paper studies n-step returns in off-policy RL and intro- duces a novel algorithm which adapts the return’s horizon n in function of a notion of policy’s age.]←ABSTRACT [Over- all, the reviewers found that the paper presents interesting ob- servations and promising experimental results.]←STRENGTH [However, they also raised concerns in their initial reviews, regarding the clarity of the paper, its theoretical foundations and its positioning (notably regarding the bias/variance tradeoff of uncorrected n-step returns) and parts of the experimental results. ]←WEAKNESS [In the absence of rebuttal or revised manuscript from the authors, not much discussion was trig- gered.]←REBUTTAL PROCESS [Based on the initial reviews, the AC cannot recommend accepting this paper, but the au- thors are encouraged to pursue this interesting research direc- tion.]←DECISION Table 1: An example of annotated meta-review. CATE- GORY indicates the category of each sentence. into three clusters: more-to-less, less-to-more, and neck-to-neck. The more-to-less text generation …

Content truncated to 500 words