TIMEDIAL: Temporal Commonsense Reasoning in Dialog

8011

Authors

nan

Abstract

AbstractEveryday conversations require understanding everyday events, which in turn, requires understanding temporal commonsense concepts interwoven with those events. Despite recent progress with massive pre-trained language models (LMs) such as T5 and GPT-3, their capability of temporal reasoning in dialogs remains largely under-explored. In this paper, we present the first study to investigate pre-trained LMs for their temporal reasoning capabilities in dialogs by introducing a new task and a crowd-sourced English challenge set, TimeDial. We formulate TimeDial as a multiple choice cloze task with over 1.1K carefully curated dialogs. Empirical results demonstrate that even the best performing models struggle on this task compared to humans, with 23 absolute points of gap in accuracy. Furthermore, our analysis reveals that the models fail to reason about dialog context correctly; instead, they rely on shallow cues based on existing temporal patterns in context, motivating future research for modeling temporal concepts in text and robust contextual reasoning about them. The dataset is publicly available at https://github.com/google-research-datasets/timedial.

Keywords

nan

Full Content

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 658–670 August 1–6, 2021. ©2021 Association for Computational Linguistics 658 SocAoG: Incremental Graph Parsing for Social Relation Inference in Dialogues Liang Qiu1, Yuan Liang2, Yizhou Zhao1, Pan Lu1, Baolin Peng3, Zhou Yu4, Ying Nian Wu1, Song-Chun Zhu1 1UCLA Center for Vision, Cognition, Learning, and Autonomy 2University of California, Los Angeles 3Microsoft Research, Redmond 4University of California, Davis liangqiu@ucla.edu Abstract Inferring social relations from dialogues is vi- tal for building emotionally intelligent robots to interpret human language better and act ac- cordingly. We model the social network as an And-or Graph, named SocAoG, for the con- sistency of relations among a group and lever- aging attributes as inference cues. Moreover, we formulate a sequential structure prediction task, and propose an α–β–γ strategy to incre- mentally parse SocAoG for the dynamic infer- ence upon any incoming utterance: (i) an α process predicting attributes and relations con- ditioned on the semantics of dialogues, (ii) a β process updating the social relations based on related attributes, and (iii) a γ process up- dating individual’s attributes based on interper- sonal social relations. Empirical results on Di- alogRE and MovieGraph show that our model infers social relations more accurately than the state-of-the-art methods. Moreover, the abla- tion study shows the three processes comple- ment each other, and the case study demon- strates the dynamic relational inference.1 1 Introduction Social relations form the basic structure of our so- ciety, defining not only our self-images but also our relationships (Sztompka, 2002). Robots with a higher emotional quotient (EQ) have the potential to understand users’ social relations better and act appropriately. Given a dialogue as context and a set of entities, the task of Dialogue Relation Extraction (DRE) predicts the relation types between the enti- ties from a predefined relation set. Table 5 shows such an example from the dataset DialogRE (Yu et al., 2020). Existing researches using BERT-based mod- els (Devlin et al., 2018; Yu et al., 2020; Xue et al., 2020a) or graph-based models (Xue et al., 2020b; 1The code is released at https://github.com/ Liang-Qiu/SocAoG-dialogues. S1: Well then we’ll-we’ll see you the day after tomorrow. Mom?! Dad?! What-what...what you guys doing here?! S2: Well you kids talk about this place so much, we thought we’d see what all the fuss is about. S3: I certainly see what the girls like coming here. S1: Why?! S3: The sexy blonde behind the counter. S1: Gunther?! S2: Your mother just added him to her list. S1: What? Your-your list? Argument Pair Trigger Relation Type R1 (S2, S1) dad per:children R2 (S3, Gunther) sexy blonde per:positive impression R3 (S3, S1) mom per:children R4 (S1, S3) mom per:parents R5 (S1, S2) dad per:parents Table 1: A dialogue example from DialogRE (Yu et al., 2020). Trigger word annotations are not used for train- ing, but rather for illustrating purpose only. Chen et al., 2020) focus on identifying entities’ relations from …

Content truncated to 500 words

Labeled Text

AbstractEveryday conversations require understanding everyday events, which in turn, requires understanding temporal commonsense concepts interwoven with those events. Despite recent progress with massive pre-trained language models (LMs) such as T5 and GPT-3, their capability of temporal reasoning in dialogs remains largely under-explored. In this paper, we present the first study to investigate pre-trained LMs for their temporal reasoning capabilities in dialogs by introducing a new task and a crowd-sourced English challenge set, TimeDial. We formulate TimeDial as a multiple choice cloze task with over 1.1K carefully curated dialogs. Empirical results demonstrate that even the best performing models struggle on this task compared to humans, with 23 absolute points of gap in accuracy. Furthermore, our analysis reveals that the models fail to reason about dialog context correctly; instead, they rely on shallow cues based on existing temporal patterns in context, motivating future research for modeling temporal concepts in text and robust contextual reasoning about them. The dataset is publicly available at

Publication Details

Paper ID:
8011
Year:
2021
Publication:
TIPSTER
Publication Type:
conf
DOI:
nan