Semi Supervised Preposition-Sense Disambiguation using Multilingual Data

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2718–2729, Osaka, Japan, December 11-17 2016. Semi Supervised Preposition-Sense Disambiguation using Multilingual Data Hila Gonen Department of Computer Science Bar-Ilan University hilagnn@gmail.com Yoav Goldberg Department of Computer Science Bar-Ilan University yoav.goldberg@gmail.com Abstract Prepositions are very common and very ambiguous, and understanding their sense is critical for understanding the meaning of the sentence. Supervised corpora for the preposition-sense disambiguation task are small, suggesting a semi-supervised approach to the task. We show that signals from unannotated multilingual data can be used to improve supervised preposition- sense disambiguation. Our approach pre-trains an LSTM encoder for predicting the translation of a preposition, and then incorporates the pre-trained encoder as a component in a supervised classification system, and fine-tunes it for the task. The multilingual signals consistently improve results on two preposition-sense datasets. 1 Introduction Preposition-sense disambiguation (Litkowski and Hargraves, 2005; Litkowski and Hargraves, 2007; Schneider et al., 2015; Schneider et al., 2016), is the task of assigning a category to a preposition in context (see Section 2.1). Choosing the correct sense of a preposition is crucial for understanding the meaning of the text. This important semantic task is especially challenging from a learning perspective as only little amounts of annotated training data are available for it. Indeed, previous systems (see Sec- tions 2.1.1 and 5.4) make extensive use of the vast and human-curated WordNet lexicon (Miller, 1995) in order to compensate for the small size of the annotated data and obtain good accuracies. Instead, we propose to deal with the scarcity of annotated data by taking a semi-supervised approach. We rely on the intuition that word ambiguity tends to differ between languages (Dagan et al., 1991), and show that multilingual corpora can provide a good signal for the preposition sense disambiguation task. Multilingual corpora are vast and relatively easy to obtain (Resnik and Smith, 2003; Koehn, 2005; Steinberger et al., 2006), making them appealing candidates for use in a semi-supervised setting. Our approach (Section 4) is based on representation learning (Bengio et al., 2013), and can also be seen as an instance of multi-task (Caruana, 1997), or transfer learning (Pan and Yang, 2010). First, we train an LSTM-based neural network (Hochreiter and Schmidhuber, 1997) to predict a foreign (say, French) preposition given the context of an English preposition. This trains the network to map con- texts of English prepositions to representations that are predictive of corresponding foreign prepositions, which are in turn correlated with preposition senses. The learned mapper, which takes into account large amounts of parallel text, is then incorporated into a monolingual preposition-sense disambiguation sys- tem (Section 3) and is fine-tuned based on the small amounts of available supervised data. We show that the multilingual signal is effective for the preposition-sense disambiguation task on two different datasets (Section 5). This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. License details: http://creativecommons.org/licenses/by-sa/4.0/ 2718 2 Background 2.1 Preposition Sense Disambiguation Prepositions are very common, very ambiguous and …

Content truncated to 500 words