BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1 - 11 July 10-15, 2022 ©2022 Association for Computational Linguistics Social Norms Guide Reference Resolution Mitchell Abrams and Matthias Scheutz Human-Robot Interaction Laboratory, Tufts University, Medford, MA 02155 {mitchell.abrams, matthias.scheutz}@tufts.edu Abstract Humans use natural language, vision, and con- text to resolve referents in their environment. While some situated reference resolution is triv- ial, ambiguous cases arise when the language is underspecified or there are multiple candidate referents. This study investigates how prag- matic modulators external to the linguistic con- tent are critical for the correct interpretation of referents in these scenarios. In particular, we demonstrate in a human subjects experiment how the social norms applicable in the given context influence the interpretation of referring expressions. Additionally, we highlight how current coreference tools in natural language processing fail to handle these ambiguous cases. We also briefly discuss the implications of this work for assistive robots which will routinely need to resolve referents in their environment. 1 Introduction Humans interacting in natural language need to resolve referential expressions often referring to referents in their environment; utterances like pick up the green box or pick it up, for instance, high- light some referring expressions that point to a ref- erent. These expressions appear in various forms, from clear and specific—the green box—to under- specified and ambiguous—it. But reference res- olution, especially situated reference resolution, also requires vision and pragmatic context to dis- ambiguate references. In the linguistically under- specifed example of pick it up, a listener may have to look for the candidate objects in the environment to figure out what it refers to. Additionally, the so- cial setting can modulate what referent is intended, given the same referring expression and objects in the environment; in a dining room, for instance, a spoon on the ground may be the more likely candi- date than a pencil. In this paper we investigate the role of pragmatic modulators like this in reference resolution. The psycholinguistics literature has leveraged eye tracking to infer what referents humans resolve in various contexts (Tanenhaus et al., 1995; Spivey et al., 2001). Pragmatic modulators outside of the linguistic content can further constrain the referen- tial domain and affect referent interpretation, such as task-relevant constraints (Hanna and Tanenhaus, 2004). There is a gap, however, in understanding how other pragmatic modulators, such as social norms and conventions affect the interpretation of referents. For example, while there has been work on modeling what social norms are activated in various contexts and settings (Malle et al., 2020), it is unclear how norms guide humans to interpret referring expressions. Similarly, conventions such as standing on the right side of an escalator while walking on the left, or sitting in the back of cab, can have modulatory influence on reference resolution and object selection. The aim of this paper is to demonstrate the role of pragmatic modulators, especially social norms, in guiding situated reference resolution. …

Content truncated to 500 words