Reference Matters: Benchmarking Factual Error Correction for Dialogue Summarization with Fine-grained Evaluation Framework

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 1387–1400 July 9-14, 2023 ©2023 Association for Computational Linguistics MS-DETR: Natural Language Video Localization with Sampling Moment-Moment Interaction Jing Wang1,3,4, Aixin Sun2∗, Hao Zhang1, Xiaoli Li1,3,4∗ 1 School of Computer Science and Engineering, Nanyang Technological University, Singapore 2 S-Lab, Nanyang Technological University, Singapore 3 Institute for Infocomm Research, A*STAR, Singapore 4 Centre for Frontier AI Research, A*STAR, Singapore {jing005@e.,axsun@,hao007@e.}ntu.edu.sg, xlli@i2r.a-star.edu.sg Abstract Given a query, the task of Natural Language Video Localization (NLVL) is to localize a tem- poral moment in an untrimmed video that se- mantically matches the query. In this paper, we adopt a proposal-based solution that generates proposals (i.e., candidate moments) and then select the best matching proposal. On top of modeling the cross-modal interaction between candidate moments and the query, our proposed Moment Sampling DETR (MS-DETR) enables efficient moment-moment relation modeling. The core idea is to sample a subset of mo- ments guided by the learnable templates with an adopted DETR (DEtection TRansformer) framework. To achieve this, we design a multi- scale visual-linguistic encoder, and an anchor- guided moment decoder paired with a set of learnable templates. Experimental results on three public datasets demonstrate the superior performance of MS-DETR.1 1 Introduction Natural language video localization (NLVL) aims to retrieve a temporal moment from an untrimmed video that semantically corresponds to a given lan- guage query, see Fig. 1 for an example. This task is also known as temporal sentence grounding in video, and video moment retrieval. As a fundamen- tal video-language task, it has a wide range of ap- plications, such as video question answering (Fan et al., 2019; Yu et al., 2018; Li et al., 2019), video retrieval (Gabeur et al., 2020; Liu et al., 2019; Chen et al., 2020), and video grounded dialogue (Le et al., 2019; Kim et al., 2021). Generally speaking, in NLVL models, a video is first split to a sequence of many small fixed-length segments. Video features are then extracted from these segments to interact with the text query. Con- ceptually, each video segment can be viewed as a ∗Corresponding Authors 1Code is released in https://github.com/K-Nick/MS-DETR Video Ground-Truth Query He stops playing and continues talking. Moment Candidate #1 Moment Candidate #2 Figure 1: An NLVL example with query and ground truth video moment. Two moment candidates with similar video features are also highlighted in light and dark green colors. form of “video token”. There are mainly two gen- res of approaches to NLVL. Proposal-free methods directly model the interaction between video tokens and text, and aim to identify start/end boundaries along the video token sequence. Proposal-based methods generate candidate moments as propos- als and then select the best matching proposal2 as the answer. Each proposal is a continuous span of video tokens. To generate proposals, some methods enumer- ate all possible moment candidates via pre-defined anchors. Anchors are reference start/end positions along the video. Fig. 2 shows three 2D-Map exam- ples. Each cell in …

Content truncated to 500 words