A multi-constraint structured hinge loss for named-entity recognition

A multi-constraint structured hinge loss for named-entity recognition Hanieh Poostchi Nine Entertainment Co. Willoughby NSW 2068, Australia HPoostchi@nine.com.au Massimo Piccardi FEIT, University of Technology Sydney Broadway NSW 2007, Australia Massimo.Piccardi@uts.edu.au Abstract The negative log-likelihood or cross entropy is the usual training objective of NLP mod- els owing to its versatility and empirical per- formance. However, training objectives which directly target the performance measure used to evaluate the task have the potential to lead to higher empirical accuracy. For this rea- son, in this short paper we propose using a multi-constraint structured hinge loss as the training objective of a contemporary named- entity recognition (NER) model. Experimen- tal results over the challenging OntoNotes 5.0 dataset have shown that the proposed objective has been able to achieve an improvement of 0.62 CoNLL score points at a complete parity of testing set-up. 1 Introduction All NLP models utilise a loss function as minimi- sation objective for model training. Choosing the most appropriate loss function for a particular task can play an important role in the performance of the trained models at test time and in-field. How- ever, almost invariably the utilised loss function is the negative log-likelihood (NLL), also known as cross entropy. This is due to a number of at- tractive properties of the NLL such as its smooth- ness and differentiability in large regions of the parameter space. In addition, training with min- imum NLL often leads to models of high empiri- cal accuracy. However, this function is not exempt from shortcomings. To name two, 1) the NLL only rewards the probability of the ground-truth class and does not distinguish between the other classes, and 2) it does not impose explicit mar- gins (or ratios) between the probability assigned to the ground-truth class and those assigned to the other classes. For this reason, other differentiable loss functions are regarded as appealing alterna- tives or complements to the NLL. Amongst them are the hinge loss (Cortes and Vapnik, 1995) and the REINFORCE loss (Williams, 1992; Ranzato et al., 2016) which both attempt to directly op- timise the performance measure used to evaluate the model’s accuracy (e.g., the Hamming loss, the CoNLL score, the BLEU score etc). Both these losses can be used for the usual classification at token level or for the joint classification of all the tokens in a sentence (i.e., structured prediction) (Tsochantaridis et al., 2005). Given that targeting the evaluation loss during training may lead to im- proved performance at test time, in this short paper we explore the use of a structured hinge loss for named-entity recognition (NER). Our main contri- bution is the introduction of additional constraints between specific labelings aimed at increasing the accuracy of the learned model. Experimental re- sults over a challenging NER dataset (OntoNotes 5.0, which is still far from accuracy saturation) show that the proposed approach has been able to achieve higher accuracy than both the NLL and a conventional structured hinge loss. 2 Related Work In this section we briefly review …

Content truncated to 500 words