Addressing Slot-Value Changes in Task-oriented Dialogue Systems through Dialogue Domain Adaptation

20250

Authors

nan

Abstract

AbstractRecent task-oriented dialogue systems learn a model from annotated dialogues, and such dialogues are in turn collected and annotated so that they are consistent with certain domain knowledge. However, in real scenarios, domain knowledge is subject to frequent changes, and initial training dialogues may soon become obsolete, resulting in a significant decrease in the model performance. In this paper, we investigate the relationship between training dialogues and domain knowledge, and propose Dialogue Domain Adaptation, a methodology aiming at adapting initial training dialogues to changes intervened in the domain knowledge. We focus on slot-value changes (e.g., when new slot values are available to describe domain entities) and define an experimental setting for dialogue domain adaptation. First, we show that current state-of-the-art models for dialogue state tracking are still poorly robust to slot-value changes of the domain knowledge. Then, we compare different domain adaptation strategies, showing that simple techniques are effective to reduce the gap between training dialogues and domain knowledge.

Keywords

nan

Full Content

Proceedings of Recent Advances in Natural Language Processing, pages 63–72 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_008 63 Comparing Supervised Machine Learning Techniques for Genre Analysis in Software Engineering Research Articles Felipe A. de Britto1, Thiago C. Ferreira2, Leonardo P. Nunes2, and Fernando S. Parreiras1 FUMEC University1 Federal University of Minas Gerais2 felipebritto@gmail.com, thiagocf05@ufmg.br, leopereiranunes@ufmg.br, fernando.parreiras@fumec.br Abstract Written communication is of utmost impor- tance to the progress of scientific research. The speed of such development, however, may be affected by the scarcity of reviewers to ref- eree the quality of research articles. In this context, automatic approaches that are able to query linguistic segments in written contribu- tions by detecting the presence or absence of common rhetorical patterns have become a ne- cessity in the refereeing process. This paper aims to compare supervised machine learning techniques tested to accomplish genre analysis in Introduction sections of software engineer- ing articles. A semi-supervised approach to augment the number of annotated sentences in SciSents1 was performed. Two supervised ap- proaches using SVM and logistic regression to assess the F-score for genre analysis in the cor- pus were undertaken. A technique based on logistic regression and BERT has been found to perform genre analysis highly satisfactorily with an average of 88.25 on F-score when re- trieving patterns at an overall level. 1 Introduction Written communication plays a fundamental role in scholarly development. Evidence for this is the high number of estimated publications and jour- nals (Larsen and von Ins, 2010; Bj¨ork et al., 2008; Mabe, 2003). In this scenario, the reviewing pro- cess is a crucial pathway for improving publication quality, as it acts as a filter through which suitable research papers are selected for publication (Ware and Mabe, 2015). In principle, although academic gatekeeping does not entail rigid language rubrics, scientific publications on the whole present stan- dardised conventions such as preference for passive constructions, high nominal style, paper division 1Avaliable on: https://github.com/coling2020- lais/SciSents in sections, and use of lexical and phrasal struc- tures to indicate the function and purpose of each text portion (Seaghdha and Teufel, 2014). Dis- seminated linguistic work aiming to systematically describe writing organization with a focus on the In- troduction section is the CARS (Create a Research Space) model (Swales, 1990). CARS approaches genre analysis by introducing two concepts, namely Moves and Steps. Whereas a Move represents the objectives and functions of a text segment at an overall level, a Step further elaborates on explain- ing how the rhetorical means are specifically used to perform the function of a Move (Ruiying and Allison, 2003) (see examples in Table 1). Despite models serving as a basis for the reviewing pro- cess (e.g. CARS), the availability of reviewers to evaluate scientific publications does not keep pace with the ever-growing number of papers which re- quire gatekeeping (Fox, 2017), therefore making computational techniques necessary. Computational approaches may be implemented to query linguistic segments automatically in re- search articles by indicating the presence or ab- sence of commonly used rhetorical patterns. Auto- matic approaches such …

Content truncated to 500 words

Labeled Text

AbstractRecent task-oriented dialogue systems learn a model from annotated dialogues, and such dialogues are in turn collected and annotated so that they are consistent with certain domain knowledge. However, in real scenarios, domain knowledge is subject to frequent changes, and initial training dialogues may soon become obsolete, resulting in a significant decrease in the model performance. In this paper, we investigate the relationship between training dialogues and domain knowledge, and propose Dialogue Domain Adaptation, a methodology aiming at adapting initial training dialogues to changes intervened in the domain knowledge. We focus on slot-value changes (e.g., when new slot values are available to describe domain entities) and define an experimental setting for dialogue domain adaptation. First, we show that current state-of-the-art models for dialogue state tracking are still poorly robust to slot-value changes of the domain knowledge. Then, we compare different domain adaptation strategies, showing that simple techniques are effective to reduce the gap between training dialogues and domain knowledge.

Publication Details

Paper ID:
20250
Year:
2021
Publication:
TIPSTER
Publication Type:
conf
DOI:
nan