Weighted maximum likelihood loss as a convenient shortcut to optimizing the F-measure of maximum entropy classifiers

Proceedings of Recent Advances in Natural Language Processing, pages 207–214, Hissar, Bulgaria, 7-13 September 2013. Weighted maximum likelihood as a convenient shortcut to optimize the F-measure of maximum entropy classifiers Georgi Dimitroff, Laura Tolos¸i, Borislav Popov and Georgi Georgiev Ontotext AD, Sofia, Bulgaria georgi.dimitrov, laura.tolosi, borislav.popov, georgi.georgiev@ontotext.com Abstract We link the weighted maximum entropy and the optimization of the expected Fβ- measure, by viewing them in the frame- work of a general common multi-criteria optimization problem. As a result, each solution of the expected Fβ-measure max- imization can be realized as a weighted maximum likelihood solution - a well un- derstood and behaved problem. The spe- cific structure of maximum entropy mod- els allows us to approximate this charac- terization via the much simpler class-wise weighted maximum likelihood. Our ap- proach reveals any probabilistic learning scheme as a specific trade-off between dif- ferent objectives and provides the frame- work to link it to the expected Fβ-measure. 1 Introduction In many NLP classification applications, the classes are not symmetric and the user has some preference towards a high Precision or Recall of a particular target class. Thus, appropriate tun- ing of the model is often necessary, depending on the particular tolerance of the application to false positive or false negative results. This pref- erence can be expressed by requiring a large Fβ measure for a particular β describing the desired Precision/Recall trade-off. Ideally, the parameters of the linear model should be estimated such that a desired Fβ measure is maximized. However, directly maximizing Fβ is hard, due to its non- concave shape. Maximum likelihood-based classifiers such as the maximum entropy are relatively easy to fit, but they are rigid and cannot be tuned to a de- sired Precision and Recall trade-off. In this ar- ticle, we consider a more flexible maximum en- tropy model, which optimizes a weighted likeli- hood function. If appropriate weights are chosen, then the maximum weighted likelihood model co- incides with the optimal Fβ model. The advantage of the weighted likelihood as a loss function is that it is concave and standard gradient methods can be used for its optimization. In fact an existing maxi- mum entropy implementation can be easily gener- alized to the weighted case. To the best of our knowledge, such a link be- tween the maximum likelihood and the Fβ has not been established before. The article is focused on the intuition of the relation and the sketch of the proof of the main result. We also present numeri- cal experiment supporting the theoretical findings. Additional value of our theoretical observation is that it establishes the methodology of viewing a particular probabilistic model as a specific solu- tion of a common multi-criteria optimization prob- lem. This article is organized as follows. In Sec- tion 2 we present related work, Sections 3 to 6 present the theoretical aspects of link between the weighted maxent and F measure. Section 7 intro- duces the algorithm, Section 8 explains the steps for evaluation of the algorithm, Section …

Content truncated to 500 words