SWAT: Cross-Lingual Lexical Substitution using Local Context Matching, Bilingual Dictionaries and Machine Translation

Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010, pages 123–128, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics SWAT: Cross-Lingual Lexical Substitution using Local Context Matching, Bilingual Dictionaries and Machine Translation Richard Wicentowski, Maria Kelly, Rachel Lee Department of Computer Science Swarthmore College Swarthmore, PA 19081 USA richardw@cs.swarthmore.edu, {mkelly1,rlee1}@sccs.swarthmore.edu Abstract We present two systems that select the most appropriate Spanish substitutes for a marked word in an English test sen- tence. These systems were official en- tries to the SemEval-2010 Cross-Lingual Lexical Substitution task. The first sys- tem, SWAT-E, finds Spanish substitutions by first finding English substitutions in the English sentence and then translating these substitutions into Spanish using an English-Spanish dictionary. The second system, SWAT-S, translates each English sentence into Spanish and then finds the Spanish substitutions in the Spanish sen- tence. Both systems exceeded the base- line and all other participating systems by a wide margin using one of the two official scoring metrics. 1 Introduction We present two systems submitted as official en- tries to the SemEval-2010 Cross-Lingual Lexical Substitution task (Mihalcea et al., 2010). In this task, participants were asked to substitute a single marked word in an English sentence with the most appropriate Spanish translation(s) given the con- text. On the surface, our two systems are very sim- ilar, performing monolingual lexical substitution and using translation tools and bilingual dictionar- ies to make the transition from English to Spanish. 2 Scoring The task organizers used two scoring metrics adapted from the SemEval-2007 English Lexical Substitution task (McCarthy and Navigli, 2007). For each test item i, human annotators provided a multiset of substitutions, Ti, that formed the gold standard. Given a system-provided multiset an- swer Si for test item i, the best score for a sin- gle test item is computed using (1). Systems were allowed to provide an unlimited number of re- sponses in Si, but each item’s best score was di- vided by the number of answers provided in Si. best score = � s∈Si frequency(s ∈ Ti) |Si| · |Ti| (1) The out-of-ten score, henceforth oot, limited sys- tems to a maximum of 10 responses for each test item. Unlike the best scoring method, the final score for each test item in the oot method is not di- vided by the actual number of responses provided by the system; therefore, systems could maximize their score by always providing exactly 10 re- sponses. In addition, since Si is a multiset, the 10 responses in Si need not be unique. oot score = � s∈Si frequency(s ∈ Ti) |Ti| (2) Further details on the oot scoring method and its impact on our systems can be found in Section 3.4. The final best and oot score for the system is computed by summing the individual scores for each item and, for recall, dividing by the number of tests items, and for precision, dividing by the number of test items answered. Our systems pro- vided a response to every test item, so …

Content truncated to 500 words