Rewarding Coreference Resolvers for Being Consistent with World Knowledge

Rewarding Coreference Resolvers for Being Consistent with World Knowledge Rahul Aralikatte1, Heather Lent1, Ana Valeria Gonzalez1, Daniel Hershcovich1, Chen Qiu1,2, Anders Sandholm3, Michael Ringaard3, and Anders Søgaard1,3 1University of Copenhagen, 2China University of Geosciences, 3Google Research {rahul,hcl,ana,hershcovich,chen.qiu,soegaard}@di.ku.dk, {sandholm,ringgaard}@google.com Abstract Unresolved coreference is a bottleneck for re- lation extraction, and high-quality coreference resolvers may produce an output that makes it a lot easier to extract knowledge triples. We show how to improve coreference resolvers by forwarding their input to a relation extraction system and reward the resolvers for produc- ing triples that are found in knowledge bases. Since relation extraction systems can rely on different forms of supervision and be biased in different ways, we obtain the best perfor- mance, improving over the state of the art, us- ing multi-task reinforcement learning. 1 Introduction Coreference annotations are costly and difficult to obtain, since trained annotators with sufficient world knowledge are necessary for reliable anno- tations. This paper presents a way to simulate an- notators using reinforcement learning. To moti- vate our approach, we rely on the following exam- ple from Martschat and Strube (2014, colors added to mark entity mentions): (1) [ ........ Lynyrd.......... Skynyrd]1 was formed in Florida2. Other bands from [the Sunshine State]2 in- clude Fireflight and :::::::: Marilyn:::::::: Manson. Martschat and Strube (2014) cite the associa- tion between Florida and the Sunshine State as an example of a common source of name-name re- call error for state-of-the-art coreference resolu- tion systems. The challenge is that the two names co-occur relatively infrequently and are unlikely to do so in a moderate-sized, manually annotated training corpus. A state-of-the-art system may be able to infer the relation using distributional infor- mation about the phrase the Sunshine State, but is likely to have limited evidence for the decision that it is coreferential with Florida rather than ....... Lynyrd ......... Skynyrd. Wikipedia documents Resolved documents Coreference resolver Policy gradient Rewards SRO triples extraction Triples scorers Figure 1: Our strategy for training a coreference re- solver using reward from relation extraction. While coreference-annotated data is scarce, knowledge bases including factual information (such as that Fireflight is from Florida) are in- creasingly available. For a human annotator un- aware that Florida is sometimes referred to as the Sunshine State, the information that Fireflight is from Florida is sufficient to establish that Florida and the Sunshine State are (with high probability) coreferential. This paper explores a novel archi- tecture for making use of such information from knowledge bases by tying a coreference resolution system to a relation extraction system, enabling us to reward the coreference system for making predictions that lead us to infer facts that are con- sistent with such knowledge bases. This poten- tially provides us with more evidence for resolving coreference such as (1). We propose a training strategy (Figure 1) in which we pass on the predictions of a neural coreference resolver to an open relation extraction (OpenRE) system, matching relations extracted from resolved sentences with a knowledge base. We show how …

Content truncated to 500 words