Don’t understand a measure? Learn it: Structured Prediction for Coreference Resolution optimizing its measures

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1018–1028 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1094 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1018–1028 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1094 Don’t understand a measure? Learn it: Structured Prediction for Coreference Resolution optimizing its measures Iryna Haponchyk∗ and Alessandro Moschitti ∗DISI, University of Trento, 38123 Povo (TN), Italy Qatar Computing Research Institute, HBKU, 34110, Doha, Qatar {gaponchik.irina,amoschitti}@gmail.com Abstract An assential aspect of structured predic- tion is the evaluation of an output struc- ture against the gold standard. Especially in the loss-augmented setting, the need of finding the max-violating constraint has severely limited the expressivity of effec- tive loss functions. In this paper, we trade off exact computation for enabling the use of more complex loss functions for coreference resolution (CR). Most note- worthily, we show that such functions can be (i) automatically learned also from controversial but commonly accepted CR measures, e.g., MELA, and (ii) success- fully used in learning algorithms. The ac- curate model comparison on the standard CoNLL–2012 setting shows the benefit of more expressive loss for Arabic and En- glish data. 1 Introduction In recent years, interesting structured predic- tion methods have been developed for coref- erence resolution (CR), e.g., (Fernandes et al., 2014; Bj¨orkelund and Kuhn, 2014; Martschat and Strube, 2015). These models are supposed to out- put clusters but, to better control the exponential nature of the problem, the clusters are converted into tree structures. Although this simplifies the problem, optimal solutions are associated with an exponential set of trees, requiring to maximize over such a set. This originated latent models (Yu and Joachims, 2009) optimizing the so-called loss- augmented objective functions. In this setting, loss functions need to be factor- izable together with the feature representations for finding the max-violating constraints. The conse- quence is that only simple loss functions, basically just counting incorrect edges, were applied in pre- vious work, giving up expressivity for simplicity. This is a critical limitation as domain experts con- sider more information than just counting edges. In this paper, we study the use of more ex- pressive loss functions in the structured predic- tion framework for CR, although some findings are clearly applicable to more general settings. We attempted to optimize the complicated offi- cial MELA measure1 (Pradhan et al., 2012) of CR within the learning algorithm. Unfortunately, MELA is the average of measures, among which CEAFe has an excessive computational complex- ity preventing its direct use. To solve this prob- lem, we defined a model for learning MELA from data using a fast linear regressor, which can be then effectively used in structured prediction al- gorithms. We defined features to learn such a loss function, e.g., different link counts or aggregations such as Precision and Recall. Moreover, we de- signed methods for generating training data from which our regression loss algorithm (RL) …

Content truncated to 500 words