Investigating Lexical Substitution Scoring for Subtitle Generation

Proceedings of the 10th Conference on Computational Natural Language Learning (CoNLL-X), pages 45–52, New York City, June 2006. c⃝2006 Association for Computational Linguistics Investigating Lexical Substitution Scoring for Subtitle Generation Oren Glickman and Ido Dagan Computer Science Department Bar Ilan University Ramat Gan, Israel {glikmao,dagan}@cs.biu.ac.il Mikaela Keller and Samy Bengio IDIAP Research Institute Martigny, Switzerland {mkeller,bengio}@idiap.ch Walter Daelemans CNTS Antwerp, Belgium walter.daelemans@ua.ac.be Abstract This paper investigates an isolated setting of the lexical substitution task of replac- ing words with their synonyms. In par- ticular, we examine this problem in the setting of subtitle generation and evaluate state of the art scoring methods that pre- dict the validity of a given substitution. The paper evaluates two context indepen- dent models and two contextual models. The major findings suggest that distribu- tional similarity provides a useful comple- mentary estimate for the likelihood that two Wordnet synonyms are indeed substi- tutable, while proper modeling of contex- tual constraints is still a challenging task for future research. 1 Introduction Lexical substitution - the task of replacing a word with another one that conveys the same meaning - is a prominent task in many Natural Language Pro- cessing (NLP) applications. For example, in query expansion for information retrieval a query is aug- mented with synonyms of the original query words, aiming to retrieve documents that contain these syn- onyms (Voorhees, 1994). Similarly, lexical substi- tutions are applied in question answering to identify answer passages that express the sought answer in different terms than the original question. In natu- ral language generation it is common to seek lex- ical alternatives for the same meaning in order to reduce lexical repetitions. In general, lexical sub- stitution aims to preserve a desired meaning while coping with the lexical variability of expressing that meaning. Lexical substitution can thus be viewed within the general framework of recognizing entail- ment between text segments (Dagan et al., 2005), as modeling entailment relations at the lexical level. In this paper we examine the lexical substitu- tion problem within a specific setting of text com- pression for subtitle generation (Daelemans et al., 2004). Subtitle generation is the task of generat- ing target language TV subtitles for video recordings of a source language speech. The subtitles should be of restricted length, which is often shorter than the full translation of the original speech, yet they should maintain as much as possible the meaning of the original content. In a typical (automated) subtitling process the original speech is first trans- lated fully into the target language and then the tar- get translation is compressed to optimize the length requirements. One of the techniques employed in the text compression phase is to replace a target lan- guage word in the original translation with a shorter synonym of it, thus reducing the character length of the subtitle. This is a typical lexical substitution task, which resembles similar operations in other text compression and generation tasks (e.g. (Knight and Marcu, 2002)). This paper investigates the task of assigning like- …

Content truncated to 500 words