Numerically Grounded Language Models for Semantic Error Correction

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 987–992, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Numerically Grounded Language Models for Semantic Error Correction Georgios P. Spithourakis and Isabelle Augenstein and Sebastian Riedel Department of Computer Science University College London {g.spithourakis, i.augenstein, s.riedel}@cs.ucl.ac.uk Abstract Semantic error detection and correction is an important task for applications such as fact checking, speech-to-text or grammatical er- ror correction. Current approaches gener- ally focus on relatively shallow semantics and do not account for numeric quantities. Our approach uses language models grounded in numbers within the text. Such groundings are easily achieved for recurrent neural lan- guage model architectures, which can be fur- ther conditioned on incomplete background knowledge bases. Our evaluation on clinical reports shows that numerical grounding im- proves perplexity by 33% and F1 for semantic error correction by 5 points when compared to ungrounded approaches. Conditioning on a knowledge base yields further improvements. 1 Introduction In many real world scenarios it is important to de- tect and potentially correct semantic errors and in- consistencies in text. For example, when clinicians compose reports, some statements in the text may be inconsistent with measurements taken from the patient (Bowman, 2013). Error rates in clinical data range from 2.3% to 26.9% (Goldberg et al., 2008) and many of them are number-based errors (Arts et al., 2002). Likewise, a blog writer may make statistical claims that contradict facts recorded in databases (Munger, 2008). Numerical concepts constitute 29% of contradictions in Wikipedia and GoogleNews (De Marneffe et al., 2008) and 8.8% of contradictory pairs in entailment datasets (Dagan et al., 2006). Figure 1: Semantic error correction using language models. “EF” is a clinical term and stands for “ejection fraction”. These inconsistencies may stem from oversight, lack of reporting guidelines or negligence. In fact they may not even be errors at all, but point to inter- esting outliers or to errors in a reference database. In all cases, it is important to spot and possibly correct such inconsistencies. This task is known as semantic error correction (SEC) (Dahlmeier and Ng, 2011). In this paper, we propose a SEC approach to sup- port clinicians with writing patient reports. A SEC system reads a patient’s structured background in- formation from a knowledge base (KB) and their clinical report. Then it recommends improvements to the text of the report for semantic consistency. An example of an inconsistency is shown in Figure 1. 987 The SEC system has been trained on a dataset of records and learnt that the phrases “non dilated” and “severely dilated” correspond to high and low val- ues for “EF” (abbreviation for “ejection fraction”, a clinical measurement), respectively. If the system is then presented with the phrase “non dilated” in the context of a low value, it will detect a seman- tic inconsistency and correct the text to “severely di- lated”. Our contributions are: 1) a straightforward ex- tension to recurrent neural network (RNN) language models for grounding them in …

Content truncated to 500 words