HMEANT

We introduce the first fully automatic, fully seman- tic frame based MT evaluation metric, MEANT, that outperforms all other commonly used auto- matic metrics in correlating with human judgment on …...

There has been a recent surge of interest in semantic machine translation, which stan- dard automatic metrics struggle to evalu- ate. A family of measures called MEANT has been proposed …...

AbstractWe present analyses showing that HMEANT is a reliable, accurate and fine-grained semantic frame based human MT evaluation metric with high inter-annotator agreement (IAA) and correlation with human adequacy judgments, …...