GRICE: A Grammar-based Dataset for Recovering Implicature and Conversational rEasoning

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2074–2085 August 1–6, 2021. ©2021 Association for Computational Linguistics 2074 GRICE: A Grammar-based Dataset for Recovering Implicature and Conversational rEasoning Zilong Zheng, Shuwen Qiu, Lifeng Fan, Yixin Zhu, Song-Chun Zhu UCLA Center for Vision, Cognition, Learning, and Autonomy {z.zheng,s.qiu,lfan,yixin.zhu}@ucla.edu, sczhu@stat.ucla.edu https://zilongzheng.github.io/Grice/ Abstract Understanding what we genuinely mean in- stead of what we literally say in conversations is challenging for both humans and machines; yet, this direction is mostly left untouched in modern open-ended dialogue systems. To fill in this gap, we present a grammar-based dia- logue dataset, GRICE, designed to bring impli- cature into pragmatic reasoning in the context of conversations. Our design of GRICE also in- corporates other essential aspects of modern dialogue modeling (e.g., coreference). The en- tire dataset is systematically generated using a hierarchical grammar model, such that each di- alogue context has intricate implicatures and is temporally consistent. We further present two tasks, the implicature recovery task followed by the pragmatic reasoning task in conversa- tion, to evaluate the model’s reasoning capa- bility. In experiments, we adopt baselines that claimed to have pragmatics reasoning capabil- ity; the results show a significant performance gap between baseline methods and human per- formance. After integrating a simple module that explicitly reasons about implicature, the model shows an overall performance boost in conversational reasoning. These observations demonstrate the significance of implicature re- covery for open-ended dialogue reasoning and call for future research in conversational impli- cature and conversational reasoning. 1 Introduction “When a diplomat says yes, he means ‘per- haps’; when he says perhaps, he means ‘no’; when he says no, he is not a diplomat.” —Voltaire, quoted in Spanish in Escandell (1996) (Korta and Perry, 2020) Voltaire’s above quote is an epitome of a cru- cial aspect of conversation; the meaning of the very same word or token varies according to its context and goes beyond what we literally say, Alice: Did you see the apples? Bob: There is a basket in the dining room. (The apples are in the dining room.) Alice: How many? Bob: There are at least two. (I am not sure how many apples are there.) Alice: Did you put them there? Bob: I was in the kitchen. (I didn’t put the apples in the dining room.) Alice: Are all the oranges there? Bob: Some are there. (Not all the oranges are in the kitchen.) Alice: What about the pears? Bob: They are in the living room. (The pears are not in the kitchen.) Figure 1: An example of the conversation in the proposed GRICE dataset. Each round of dialogue in- cludes a question, an answer that may contain implica- ture, and a recovered statement that converts the impli- cature to explicature. Different colors highlight corefer- ence flows. which is the central character of the field of prag- matics. Such a high-level comprehension of utter- ance is more than traditional semantics and logic; it is often believed to involve the construction of the speaker’s intents, …

Content truncated to 500 words