Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 432–447 June 6–11, 2021. ©2021 Association for Computational Linguistics 432 DART: Open-Domain Structured Data Record to Text Generation Linyong Nan1 Dragomir Radev1,2 Rui Zhang3 Amrit Rau1 Abhinand Sivaprasad1 Chiachun Hsieh4 Xiangru Tang1 Aadit Vyas1 Neha Verma1 Pranav Krishna5 Yangxiaokang Liu1 Nadia Irwanto1 Jessica Pan1 Faiaz Rahman1 Ahmad Zaidi1 Mutethia Mutuma1 Yasin Tarabar1 Ankit Gupta1 Tao Yu1 Yi Chern Tan1 Xi Victoria Lin2∗ Caiming Xiong2 Richard Socher2 Nazneen Fatema Rajani2 1 Yale University 2 Salesforce Research 4 The University of Hong Kong3 Penn State University 5 MIT {linyong.nan, dragomir.radev}@yale.edu, rmz5227@psu.edu, nazneen.rajani@salesforce.com Abstract We present DART, an open domain structured DAta-Record-to-Text generation dataset with over 82k instances (DARTs). Data-to-text an- notations can be a costly process, especially when dealing with tables which are the ma- jor source of structured data and contain non- trivial structures. To this end, we propose a procedure of extracting semantic triples from tables that encodes their structures by exploit- ing the semantic dependencies among table headers and the table title. Our dataset con- struction framework effectively merged hetero- geneous sources from open domain semantic parsing and spoken dialogue systems by uti- lizing techniques including tree ontology an- notation, question-answer pair to declarative sentence conversion and predicate unification, all with minimum post-editing. systematic evaluation on DARTWe present as well as new state-of-the-art results on WebNLG 2017 to show that DART (1) poses new challenges to existing data-to-text datasets and (2) facil- itates out-of-domain generalization. Our data and code can be found at https://github. com/Yale-LILY/dart. 1 Introduction Automatically generating textual descriptions from structured data improves the accessibility of knowl- edge bases to lay users. Such applications include explaining data records to non-experts (Cawsey et al., 1997), writing sports news (Chen and Mooney, 2008), summarizing information in mul- tiple documents (Fan et al., 2019), and generating dialogue responses (Wen et al., 2015). While significant progress has been made in this field, there are still several issues with existing Data-to-Text datasets. First, they adopt a flat ontol- ogy structure of the data, such as slot-value pairs for data records (Lebret et al., 2016; Novikova et al., 2017b) or flat schema for tables (Wiseman et al., ∗Now at Facebook AI. 2017; Chen et al., 2020a; Parikh et al., 2020). This flat structure is not powerful enough to encode rich semantic relationships in the ontology of the struc- tured data, especially tables, whose representation can be further improved with these semantic knowl- edge. Second, some of the datasets only focus on a small number of domains or knowledge graphs, therefore providing limited number of predicates and data ontologies. For example, E2E (Novikova et al., 2017b) on restaurants and WebNLG (Gar- dent et al., 2017) on 15 categories from DBPedia. Furthermore, some of them only have loose align- ments between data input and sentence due to the nature of the task (Wiseman et al., 2017) and the automatic generation procedure (Vougiouklis et al., 2018; …

Content truncated to 500 words